As the director of the Massachusetts Institute of Technology’s Humans and Automation Laboratory, I was asked to comment from a technologist’s perspective at the recent symposium Drone Warfare: New Robotics & Targeted Killings on the panel  “Unmanned Robotics & New Warfare.”  My perspective is unique in that not only do I conduct millions of dollars of research in the development of technologies to enable one or more humans to control unmanned vehicles (i.e., robots) more easily, but I also look at these issues from the perspective of having flown advanced fighters in the U.S. Navy, namely the F/A-18 Hornet.

While there are many forms of unmanned vehicles in use in the military today (unmanned aerial vehicles (UAVs), unmanned ground vehicles (UGVs), underwater unmanned vehicles (UUVs), etc.), my comments will focus primarily on UAVs because they are currently used to perform targeted killings.  However, it should be noted that unmanned vehicles of all types could be expected to do the same at some point in the future.

The term “control” is a misnomer when discussing UAV control because, in most settings, human operators “supervise” the control of a UAV as opposed to directly or manually controlling it.  The distinction between supervisory and manual UAV control is critical.  Supervisory control is intermittent human interaction with automated systems using high-level knowledge-based cognitive processes such as judgment and experience, whereas manual control occurs through human skill-based direct vehicle manipulation, e.g., a pilot flying a plane with a stick and rudder.  In supervisory control, humans are more on the loop than in the loop.

It is precisely this move from manual to supervisory control for UAVs that has caused and will continue to cause ripples throughout the military because the basic skills once required of manned aircraft pilots are no longer needed.  UAV control today is effectively a “click and point” paradigm, so any person familiar with a basic PC or Mac® can be a UAV “pilot”.  Indeed, in my laboratory, we recently conducted formal studies demonstrating that with just three minutes of training people with no experience flying UAVs can effectively use an iPhone® to control a UAV carrying a webcam.  These novices could manipulate the system with such precision that many could read the 20/30 line of an eye chart through the UAV’s webcam.

Automation has advanced to such a degree that pilot stick-and-rudder skills are no longer needed in UAV applications.  Due to this advancement, the U.S. Army can train UAV operators to “fly” a UAV in about 10 weeks.  However, in a clear attempt to uphold a pilot-centric culture, the Air Force still requires two years of pilot training to control the exact same vehicles with the exact same capabilities.  The primary difference between the two services is that Air Force pilots are allowed to “fly” the vehicles through joystick actions, while Army operators literally command their vehicles through point and click interactions.  Unfortunately, temporal latencies inherent in remote control of UAVs like those of U.S. Air Force Predators mean that pilots who try to manually fly UAVs introduce significantly greater risk in operations: roughly half of all Predator accidents are due to pilot error.  As a result, the Air Force has recently mandated that all take-offs and landings must be fully automated.  The Army has adhered to this policy from the beginning of its UAV operations.

While UAV control can be a cognitively “easy” task, overall UAV operations are by no means easy.  These operations have introduced a number of complex interactions that require significantly more coordination across several agencies, including de-confliction with manned aircraft.  In addition, the use of UAV full-motion video and other sources that provide information from both manned and unmanned vehicles in near real-time has overloaded decision makers.  The massive influx of electronic data from a wide array of sensors has literally left the U.S. military drowning in data.  How to distill the voluminous amount of information available in real-time to aid decision-makers in critical life-or-death decisions is still an open area of research and development, and the problem only promises to get worse before it gets better.

The possible moral, legal, and ethical implications of unmanned technologies are many, and I will discuss just a few that are specific to my area of expertise.  First, while UAVs have not really changed how warfare is conducted in terms of targeted killings (i.e., we search, then detect, then kill), the remote and nearly constant presence of UAVs has significantly shortened what is known as the “kill chain”.  Prior to UAVs and the electronic networks that accompany them, finding, identifying, and then authorizing the use of deadly force against a suspected enemy often took days or even weeks to accomplish.  Now, these same processes occur in a matter of minutes or maybe hours.  Given these compressed time schedules and the well-documented ability of people to more readily engage in the use of deadly force at a distance (a phenomenon I describe as a “moral buffer”), as well as the relatively ease of UAV control that I previously discussed, it is quite possible that we could engage in more deadly interactions without the necessary time to reflect that older airborne-based systems inherently provided.  However, it is also possible that the networks of information and the near real-time interconnectivity of decision-makers could allow us to engage in safer, more ethical engagements because lawyers, politicians, and military leadership can literally all see the same video feed at the same time and come to consensus about the “right” decision to make.

Lastly, while the focus of this Symposium was the use of unmanned technologies for targeted killings, one social impact rarely addressed that I feel will be much more compelling in years to come is the use of these technologies for acts beyond those of targeted killings.  Given that almost anyone can now control a UAV with their iPhone®, it does not take a MIT futurist to imagine a scenario where such technologies could be used in terrorist acts both overseas and on U.S. soil.  For example, a small UAV could easily be flown into any sports stadium loaded with a deadly biological agent.  Such technologies may also be used in many more insidious ways, such as monitoring a point or person of interest, as these platforms are effectively mobile cameras that can transmit over a network.  Privacy could be redefined dramatically given the future ability of a UAV (or a bug-sized UGV) to follow you almost anywhere and transmit your every action over the Internet.

Mary (Missy) Cummings is an Associate Professor in the Aeronautics & Astronautics Department at the Massachusetts Institute of Technology.  She was an officer and a pilot in the U.S. Navy: one of the Navy’s first female fighter pilots.  More information about her research at HAL can be found at halab.mit.edu.

Image courtesy of Singularity Hub.

Please click here to read as a PDF

Killer mechanical robots the size of flies, giant predator drones piloted from an iPhone, together with a new mode of warfare embraced by the U.S. military and both political parties in Washington.  That is the upshot of the recent symposium – “New Robotics and the Legality of Targeted Killings” – hosted by the Harvard National Security Journal.  The technology is here to stay, and it is being deployed to kill designated enemies of the United States and its allies.  What are the legal and ethical implications of this trend?  And what rules govern killing by pilotless drones in some of the most remote regions of the world?

Surprisingly, we seem to have no idea.

As a former official overseeing national strategy in two warzones, I appreciate how law and ethics can take a back seat to new tactics that turn the tide against committed enemies.  So long as the tactics are legally available, whatever the theory, then the tactics will be used.  In Iraq, there have probably been more Predator drone strikes than anywhere else on earth – and with tremendous effect, degrading extremist networks and decapitating leadership cells.  Drone attacks alone are not strategically sound, but when combined with a campaign to secure the population against common enemies, the strategic advantages are proven and empirical.  The same strategy is now being employed in Afghanistan.

The Obama administration has, quite rightly in my view, also increased the targeting of al Qaida and Taliban leaders in the ungoverned tribal areas of northwestern Pakistan.  Many of these areas at the moment are inaccessible to Pakistani security forces, but a longer-term campaign plan will see Pakistani forces deploying in force to secure its population.  Until that can happen however, without sustained surveillance and drone strikes, we would accept a sanctuary for terrorist cells committed to killing U.S. forces in Afghanistan and threatening the stability of Pakistan (a country with 172 million people and nuclear weapons).  Ten years ago we paid a price for leaving such a sanctuary unmolested, and no U.S. President is likely to take that risk again.  So the drones are here, and they are here to stay.

But with increasing warfare there is an increasing need to explain what we are doing to the public – and how new tactics are grounded in the rule of law.  The silence from the Obama administration in this regard is troubling and may prove to be a core weakness in an otherwise successful military program.  Even if there is little chance that a legal challenge would shut down the drone campaign, the United States could easily lose the moral high ground, which the Obama team has worked so assiduously to retain.

Indeed, without a vigorous defense from the Obama administration, the vacuum is being filled by a new and respected chorus arguing that drone attacks are illegal and, perhaps, even tantamount to murder.  Mary Ellen O’Connell, a law professor at Notre Dame, argues that drone strikes are “unlawful” under any purported theory of international law (she knocks down all of them).  Philip Alston, the United Nations’ Special Rapporteur on Extrajudicial Executions, concluded that drone attacks in the Pakistan region “may well violate international humanitarian law and international human rights law.”  The Chief Prosecutor of the International Criminal Court, Luis Mereno Ocampo, has asserted jurisdiction over all NATO and U.S. forces in Afghanistan and said the court is conducting a “preliminary investigation” of alleged war crimes committed in that theater.  “Whatever the gravest war crimes are that have been committed,” he told the Wall Street Journal, “we have to check.”[1]

The Obama team needs to get ahead of this legal train.  In warfare, nothing always goes right, and it is inevitable that a drone strike will at some point go badly awry.  Our enemies might adapt and surround themselves with children, or live in schools, using human shields to invite public scrutiny in the event of their demise.  And while drones with GPS or laser-guided munitions are among the most precise weapons in the history of warfare, targeting errors and loss of innocent life are certain.  The United States should make its case now, therefore, to justify the drone program according to international legal standards.

That framework might be humanitarian law or it might be classic self defense, as proposed by my symposium colleague, Kenneth Anderson.  But whatever the theory, what is most important is that it is articulated, well reasoned, known to the public, and vehemently defended by administration lawyers and policymakers.  Saying nothing has allowed those opposed to one of our most successful military programs define the narrative – and could leave its operators high and dry when things go wrong.

There is yet another reason to define clear standards for the drone program:  “In warfare, what comes around – goes around.” Tad Oelstrom emphasized that simple maxim during the symposium, a point driven home by MIT’s Mary Cummings, who showed with alarming detail how easily drone technology is patterned and even piloted with an iPhone.  “Yes,” she said, “there is an app for that.”  We need rules for this untraveled road now – with sober reflection and foresight – rather than in the near future, and in reaction to unforeseen events, such as drone technology in the hands of terrorists with an Xbox.

The United States is certainly the dominant player in this field at the moment, but that will change as the technology is patterned and becomes more broadly available.  Policymakers in Washington would be well served, therefore, to do everything they can to retain the technological and legal edge by establishing the norms and standards of drone warfare before it is established by the Ivory Tower – or worse – our adversaries.

Brett H. McGurk is an International Affairs Fellow at the Council on Foreign relations.  He served as Special Assistant to the President and Senior Director for Iraq and Afghanistan during the George W. Bush Administration and as a special advisor to the National Security Council under President Obama.

Image courtesy of the AP via the Huffington Post

When your favorite tool is a hammer, everything looks like a nail.  After the near-catastrophe on Northwest Airlines Flight 253 on Christmas Day, it is not surprising that many hammer away with the tools they know best: data-mining and watchlists.  The conventional wisdom is that if we know enough soon enough, we can stop the next attack.  The problem over Christmas, therefore, is identified solely as a function of that tool: we didn’t know enough, or, if we did, not everyone did, or if we all did, we forgot crucial bits in the welter of all that data—“enterprise amnesia.”

We should resist the urge to pick up a heavier hammer and expand our watchlisting efforts.  That reflex blinds us to the other tools in our toolbox, and the need for more ingenuity in using them.

Of course, sometimes only a hammer will do.  Intelligence gathering and analysis now requires massive databases and sophisticated computers.  Who could object to a “Google-like” tool to search and sort that information?  But resist the claim that you can add that feature and all will be well.  There are two problems with that approach.

First, it neglects the nature of the institutions tasked to use these tools.  The Google gap is hardly a recent revelation.  In fact, FAA officials gave the same excuse to the 9/11 Commission to explain why their proto-“No Fly List” contained only twenty names on September 11, 2001, while the State Department’s TIPOFF database listed over 61,000 names.  It was hard to make these databases interact because they were designed for different purposes, with different functions (none of which were Google-able).  The 9/11 Commission, like many blue-ribbon commissions before it, discovered that agencies do not share information well.  To quote Claude Rains in Casablanca, “I am shocked, shocked.”

Shocking, but hardly surprising.  Political institutions (and intelligence agencies are as political as any other) will always protect their turf against encroachment from competitor agencies.  That’s simply the nature of constant competition for the President’s ear and Congress’ purse.  And that’s why creating more, new institutions is not always as helpful as creating mechanisms that push existing institutions to share more.

Here is where ingenuity and creativity can play a role.  An example is the Terrorist Screening Center (TSC), one of the most important counterterrorism entities that most people have never heard of.  Notwithstanding the fact that the TSC is responsible for maintaining the No-Fly List, it received virtually no public notice in the aftermath of the Christmas Day attack.  The TSC is a multi-agency component of the FBI that operates 24/7 in an undisclosed location in Northern Virginia.  According to its director, the TSC “connects the law enforcement communities with the intelligence community by consolidating information about known and suspected terrorists into a single Terrorist Screening Database.”  That’s right, this list is supposed to access everything.  In fact, Dennis Blair’s National Counter Terrorism Center (NCTC) is supposed to dump its entire Terrorist Identities Datamart Environment into the TSC’s basket every day (and twice on Fridays).  From that mega-list, the TSC is tasked to support screening agencies with precision-crafted, smaller lists, like TSA’s No-Fly List.

That degree of funneling intelligence into a single entity is extraordinary in our fractured intelligence community.  But as we learned on Christmas Day, it was still not enough.

Although TSC is responsible for the final adjudication of nominations to its various watchlists, it is reliant on other agencies to make those initial nominations themselves—in other words, the same sharing problem that stymied cooperation before 9/11.

So Google away, but as Napoleon said, “the tools belong to the man who can use them.”  The first problem, then, is that institutions (and the problems that bedevil them) still matter.  New and bigger hammers aren’t enough without embedding pressure for competing agencies to cooperate in their use.  This is the inevitable paradox between the efficiencies of consolidation and the occasional value of built-in redundancy.

The second problem is Mr. Rosenzweig’s hammer itself.  He suggests that we should automate human intuition with data-mining tools like Admiral Poindexter’s “Total Information Awareness” system (TIA), although he regrets its “unfortunate” name.  But Total Information Awareness accurately described the idea, in much the same way that the all-seeing disembodied eye on its proponent’s symbol did.  According to official supporters at the time of the controversy, TIA meant a database “of an unprecedented scale” that would seek data on everything from cell phones to driver’s licenses to credit cards.

Tony Tether, then director of the Defense Advanced Research Products Agency (and therefore TIA’s chief defender), assured Congress that the program was designed only to give agencies the power to use “whatever data to which they currently have legal access.”  But it doesn’t take a DARPA scientist to see that “current legal access” is hardly a long-term constraint.  Once deployed, creeping pressure toward expansion is inevitable, by the same logic that argues for TIA’s adoption.

That was what was so chilling about TIA: there was no stopping point to it.  We see that pressure today with the watchlists we already have.  Why use a name-based list when biometrics are much more reliable?  Why watchlist planes but not trains?  Why permit a watchlisted person, too dangerous to fly, access to guns or chemicals?  And that’s why the focus cannot assume just “a failure of policy, not of law.”  It was the inability of TIA’s creators to identify a stopping point consistent with our society’s values that killed the program, not troubled civil libertarians (who didn’t have much traction in Congress at that time).

We owe it to ourselves to employ the best tools consistent with the values of our society.  But we also need total information awareness about how these tools change the baselines that govern our way of life.  Not everything we can do is something we should do.  The road back from whence we came runs uphill.

–Jeffrey Kahn is an Assistant Professor of Law at Southern Methodist University.  He previously served as a trial attorney in the U.S. Department of Justice, Civil Division, in Washington D.C.

Click here to read as a PDF

Homepage picture courtesy Times Online UK

It didn’t take long after 9/11 for the conventional wisdom to crystallize.  The devastating terrorist attacks were almost immediately, and almost universally, chalked up to the intelligence community’s failure to share information.  Yet if al Qaeda’s attempt to down Northwest flight 253 is any indication, the feds still haven’t learned how to connect the dots.

The Christmas plot is shaping up to be an information-sharing epic fail.  As early as last fall our spies began picking up alarming signals that something big was afoot.  The clues should have been circulating widely throughout the intelligence community.  And the system, according to former FBI counterterrorism official Ali Soufan, “should have been lighting up like a Christmas tree.”

They weren’t, and it wasn’t.

Why not?

Part of the problem is that intelligence officials have powerful incentives to hoard data from one another.  As I argue in a forthcoming law review article, information sharing threatens to undermine two of the things agencies value the most—influence and turf.

Intelligence officials want to maximize their sway over the President and his advisors.  In particular, they want the White House to rely on their judgment more than it relies on their rivals’.  And agencies know that handing over data to bureaucratic competitors can cause their influence to shrink.

If the FBI gives CIA a piece of information that turns out to be the silver bullet, CIA will get the credit for the resulting intelligence breakthrough.  From the Bureau’s perspective, that’s a pretty raw deal.  Influence is a zero sum game; if the President is listening to CIA more, he’s listening to the FBI less.

Intelligence officials also want to protect their turf, and sharing makes it harder for them to run their operations as they see fit.  If the FBI tells CIA what it’s up to, there’s a chance the Agency might muscle in and take control.

These fears may have been at work in the aftermath of the Christmas plot.  The FBI apparently decided to seek criminal charges against Umar Farouk Abdulmutallab without first consulting the Director of National Intelligence, Homeland Security, or the National Counter Terrorism Center.  One reason they may have been kept in the dark is because the Bureau feared losing control over the investigation—i.e., turf.

So what can be done?  If policymakers want to improve data exchange, they need to weaken intelligence agencies’ existing incentives to hoard and create new incentives to share.

One way to encourage good behavior would be to reward officials who routinely share information.  Exemplary employees could be promoted, handed choice assignments, and given cash rewards.  By contrast, those who continue to hoard could be reassigned to dead-end jobs, demoted, or even fired.

Another possibility would be to adopt new performance metrics.  Like professors who are up for tenure, intelligence officials could be evaluated partly on the extent to which others cite their reports.  Other analysts can’t rely on your work if they don’t know it exists, so tying career prospects to citation count would create strong pro-sharing incentives.

Finally, reformers should consider ways to restructure the incentives at the agency-wide level.  One option would be to establish a compensation scheme that allows sharing agencies to recoup at least a portion of the benefits that accrue to recipients.  If agencies can internalize some of these positive externalities, data exchange would no longer hurt their bottom lines.

Solid intelligence represents our best chance to stop the next Abdulmutallab before he sets foot on a plane.  And widespread information sharing is an essential building block of an effective intelligence system.  Yet the intelligence community isn’t going to start sharing simply because Congress and the President say please.  It’s not enough for our nation’s lawmakers to tear down the wall; intelligence agencies need to be given reasons to climb over the rubble.

–Nathan A. Sales is a law professor at George Mason University. He served in the George W. Bush administration at the Justice Department and as deputy assistant secretary of homeland security for policy.

Click here to read as a PDF

Homepage image courtesy NECN

“We slipped up.”  That’s what Patrick F. Kennedy, the Undersecretary of State for Management, said at a Senate hearing last week about the Christmas day bomb plot and the arrest of Umar Farouk Abdulmutallab.

He has a gift for understatement.

But the real question isn’t whether we “slipped up”—everyone knows we did.  It’s rather how and why we did.  The truth is that this was a failure of policy, not of law.  We did it to ourselves.  In the immediate aftermath of 9/11 the Defense Advanced Research Projects Administration (DARPA) began work on techniques of data analysis called “knowledge discovery” techniques.  They gave the project an unfortunate name—Total Information Awareness—and did a poor job of reassuring troubled civil libertarians that the program would not become a “Big Brother” tool.  The research was killed.  But the concept behind the research was visionary and those are precisely the tools that, if we had them today, would have made it more likely that we would have connected the “dots” of the  Abdulmutallab plot.  Here’s why:

The evidence is increasingly clear that the problem was not a failure of intelligence collection.  If public news reports are to be believed, it now appears that there was a reasonable amount of information about or related to Abdulmutallab.  According to the New York Times, we knew about possible underwear bombs, we knew about an unnamed Nigerian who might strike America during Christmas, and we knew from his father that Abdulmutallab had become increasingly radicalized and had been in contact with Anwar al‐Alwaki, a Yemeni radical.  We had a partial name for a terror plotter—“Umar Farouk.”  And we may even have known (though it is not yet certain) that the British had denied Abdulmutallab a visa, a boon that we, on the other hand, had granted him.

But what to do with all that information?  And why didn’t the dots get connected in the way we would have wanted?  Some, like my colleague Nathan Sales, think that there still remain institutional barriers to sharing information and that means that agencies are still hoarding data.  No doubt.

But there is a deeper and more insidious problem—one that institutionally we have yet to overcome.  The problem is that there is simply too much information out there.  And information without context is nothing but noise.  Only context and analysis transform information into knowledge and only knowledge is actionable.

Consider: The National Counter Terrorism Center (NCTC) is one of two institutions we rely on to conduct all‐source analysis of terror threats (the other is the CIA).  The NCTC’s computer systems have links to more than 30 separate government systems, with more than 80 distinct databases.  According to the Director of National Intelligence, Admiral Dennis Blair, each day the NCTC gets thousands of pieces of intelligence from around the world, reviews thousands of names, and puts 350 new names onto the watchlist—the list that Adbulmutallab was not put on.

This is a veritable flood of data.  In hindsight, of course, it is very easy to see the pieces that connect together to form a picture of Abdulmutallab’s plot.  But those 10 or so bits of information were floating in an ocean of other data—literally millions of different individual entries from thousands of different sources in a host of different databases.

Hindsight is always 20/20.  What we need is foresight.  And the problem is that we continue to fixate on a human solution to our lack of foresight.  We continue to rely on the intuition of analysts to provide the insight we need.  It is all well and good to say “with the NSA intercept about a Nigerian we should have started looking at all Nigerians” or “we should have begun looking at everyone named Umar Farouk,” but those leaps of insight and anticipation are not routine—they require analysis and consideration.  And that requires time—time to ponder the necessity of making precisely that inquiry.

But time is what our analysts don’t have.  At least not enough of it.  Not with the flood of data we are seeing.  They have to prioritize and move certain lines of inquiry to the top of the pile.  Probably, as Admiral Blair has said, the warning from Abdulmutallab’s father should have moved the “Yemen/Nigeria/Bomb” issue to the top of everyone’s pile.  But as that question moves up to the top, other intelligence questions move down.  The truth is that not all of the pieces of information rise above the noise level . . . and so long as we rely on human intuition to tell us what to pull out of the noise and what not to, we are going to make mistakes.

What we lack is not human intuition.  Rather we lack the tools to make human intuition effective and automated.  The head of the NCTC told a rather shocked Senate committee the other day that, in effect, NCTC analysts don’t have a “Google‐like” tool for database inquiries.  They can’t, for example, simply type in “Umar Farouk” and pull up all the pages with links to that name.

But even that wouldn’t be enough—because there would likely still be far too many “Umar Farouk” pages for any analyst to review (especially if instead the name we had was, for example, “Omar Abdul”).  What is necessary, as the Markle Foundation has said persistently, is for us to authorize and invest in tools that allow for automated analytics—things like tagged data (so that corrections to information are automatically transmitted for updates), identity resolution techniques (so that “Umar” and “Omar” are both considered), and persistent queries (so that a question that an analyst asked last month about Umar Farouk persists in the databases and is automatically linked to a father’s warning about his son Umar when that comes in three weeks later).  We need automated knowledge discovery systems—ones that run continuously and repeatedly so that every day we check for new information about “Umar Farouk” and about all the other hundreds of thousands of intelligence leads.  These are tasks that take time, and time is what computers have plenty of.

We don’t have those tools now.  In part it’s a question of investment and development.  But at its core it is a question of policy and politics. Without the tools needed, we develop what Jeff Jonas calls “enterprise amnesia.”  He’s right, and it’s our own fault.

All decisions have consequences.  Our decision to stop research on data analytic tools back in 2003 has led, in an almost straight line, to our analysts’ difficulty in sifting the signal of a real terrorist, like Abdulmutallab, from the noise of thousands of bits of data.  It’s time to rethink our priorities and our policies.

–Paul Rosenzweig is the Principal at Red Branch Consulting PLLC and the former Deputy Assistant Secretary for Policy at the Department of Homeland Security.

Ciick here to read as a PDF

Homepage photo courtesy The Washington Post