Spy Chief Called Silicon Valley Stooge in Army Software Civil War

Lt. Gen. Michael Flynn (right), who now serves as head of the Defense Intelligence Agency, is at the center of a vicious fight over the Army's analysis software. Photo: DoD

This is how nasty and personal the fight over the Army's wartime brain has become: Pentagon functionaries are now calling one of their own top generals a corporate stooge; the Army's supposedly independent technology testers are calling for their own reports to be destroyed; and on Thursday, Congress' top investigators demanded answers from the Defense Secretary and for all documents related to the controversy ASAP.

Under development for the better part of a decade, the $2.3 billion DCGS-A ('Distributed Common Ground System ' Army') is supposed to serve as the primary source for mining intelligence and surveillance data on the battlefield ' everything from informants' tips to satellites' images to militants' fingerprints. It's designed to be the one resource that Army intel analysts can use to find links between events, build dossiers on high-level targets, and plot out enemy attacks.

At least, that's the plan. In reality, there's a sizable contingent of troops ' including some senior officers ' who think that DCGS-A is too slow and too complicated to get the job done. Instead of DCGS ' built by defense contracting giants like Lockheed, Raytheon, and IBM ' these troops have instead begged for a data mining, information visualization, and link analysis suite from a controversial Silicon Valley firm called Palantir.

And that has the Army's defenders of DCGS-A pissed.

The trouble started in 2010 when Lt. Gen. Mike Flynn ' then the military's top intelligence officer in Afghanistan, and now the head of the Defense Intelligence Agency ' complained in a report to his fellow generals that 'the enemy is able to hide in plain slight' because 'intelligence analysts in theater currently do not have the tools required to fully analyze the tremendous amounts of information currently available.' Flynn demanded better systems for analysis. That so-called Joint Urgent Operations Needs Statement, or JUONS, was forwarded on to the Pentagon's Combating Terrorism Technical Support Office. They decided that Palantir was the answer.

The office started supplying dozens of Palantir servers to troops throughout Afghanistan ' mostly Special Forces and Marines, who never signed on to DCGS-A. The reports back were often glowing: Palantir was more intuitive to use; its interface was flashier; its dossiers were easier to share. 'Palantir reduced the time required for countless analytical functions and streamlined other, once cumbersome, processes,' Maj. Gen John Toolan, commander of the II Marine Expeditionary Force in Afghanistan, wrote in a February 12, 2012 letter. 'The innovative and collaborative capabilities of Palantir have proven their mettle and effectiveness for conventional and special operations forces in combat.'

The system tends to have that effect on military and intelligence operatives ' they either love Palantir with a white-hot passion, or curse it as useless. 'One of my investors asked me, 'Is this a company or a cult?'' company CEO Alex Karp recently recounted to Business Week.

For the Army officials trying to get the entire service on a single, sensible intelligence network, it was enough to drive them mad. Flynn's 2010 JUONS 'was clearly ghost-written by a Palantir engineer,' wrote one member of the Army's resource planning directorate in a February 24 email. Yes, a handful of units say they like Palantir. But 'they are a vocal minority,' the email added. They don't recognize Palantir's problems handling large data sets, or its problems interoperating with other intelligence databases. The only reason the Marines use the system is because 'Palantir lobbyists' pressured Congressmen to lard up the Pentagon's wartime budget with money for their data-miners. A set of Army talking points, dated July 25, reinforced the same message: 'There are multiple requests for capabilities in theater and many are ghost written by commercial vendors.'

Soldiers utilize Distributed Common Ground System-Army, or DCGS-A, operations center at Aberdeen Proving Ground, Md. Photo: U.S. Army.

Palantir has certainly been an aggressive presence around the nation's capitol. Its advertisements cover the walls of metro stations in D.C. and northern Virginia. The company spent more than half a million dollars on lobbying last year. Its 'Palantir Night Live' speakers' series in Washington has attracted luminaries from former CIA chief George Tenet to Craig Newmark, of Craigslist fame. In 2010 ' perhaps as an attempt to curry favor in D.C. ' the company agreed to participate in a cockamamie (and unsuccessful) scheme to bring down WikiLeaks. In 2012, Palantir started its own political action committee to hand out campaign donations.

Still, the reports from the field about Palantir were so positive that the Army's top officer asked the independent Army Test and Evaluation Command to survey troops, and make recommendations about how to proceed. On April 25, the Command rendered its decision: 'DCGS is overcomplicated, requires lengthy classroom instruction, and is easily perishable skill set is not used constantly.' (.pdf) Instead, the Army should 'install more Palantir servers in Afghanistan' and 'incorporate a one-week training class on Palantir' for all new intelligence analysts.

The report, signed by Brig. Gen. Laura Richardson and obtained by Danger Room, was a bombshell, shattering the DCGS-A monopoly. Less than a month later, the Army took it back. 'Please ensure that any and all copies of the 25 April report are destroyed and not distributed,' (.pdf) read an email from the command. The report was replaced with a nearly-identical document (.pdf). All that was missing was the recommendation to buy Palantir.

The replacement caused a stir when it was first reported by the Washington Times last week. And now,  Rep. Darrell Issa, chairman of the House Oversight Committee, wants an explanation. In a Thursday letter to Defense Secretary Leon Panetta (.pdf), Issa and Rep. Jason Chaffetz, chairman of the committee's national security panel, demanded 'all documents and communications referring to or relating to requests for the Palantir system' by no later than 5:00 pm on August 15, 2012.'

The Pentagon did not immediately respond. But this nasty fight over intelligence analysis could be out to get a whole lot more vicious.