zenpundit.com

JUDGMENT DAY

An interesting confluence of information has crossed my computer screen in the last 24 hours.

Fabius Maximus was kind enough to send me a PDF, “Cognitive biases potentially affecting judgment of global risks” by Eliezer Yudkowsky, Singularity Institute for Artificial Intelligence. It’s a very interesting paper on analytical thinking – or s even though a number of the points made by Yudkowsky I have seen previously made elsewhere ( the blogosphere revels in hyperactive disconfirmation biases). Their central cognitive philosophy – “….the one whom you must watch above all is yourself”, is spot on.

Secondly, over at Kent’s Imperative, one of the Kentians, let’s call them “Most Formal Prose Kent” had a highly congruent post to the Yudkowsky paper, “The sins of analytic methodologists “:

“There is an increasingly common conceit that reliance on the analyst – subject to, cognitive bias, information overload, and human fallibility – can be engineered out of the process of doing intelligence. Instead, certain methodologists would substitute organizational structures, workflow re-organization, and the introduction of supposedly superior quantitative metrics in order to create a new standard for “answers”. The underlying thrust of these efforts is to reform intelligence activities towards a more “repeatable” process, often described by industrial or scientific metaphors such “foundry” or “lab”. These typically originate from the engineering and technical intelligence disciplines, and are usually directed as criticism of typical all source efforts – particularly those grounded in social science fields or qualitative methodology.

…The fundamental flaw in many of these methodologists’ efforts is that they are essentially reductionist attempts to force the difficult and oft-times messy art of intelligence entirely into the narrow box of its scientific side. While there is a place for scientific approaches, particularly in the grounding and validation of assessment, the inherently creative, non-linear, and even non-rational elements of the profession can never be completely discarded. Most recent intelligence failures have occurred, not due to a lack of precision in judgment, but from a lack of imagination in identifying, describing, and forecasting the uncertain dynamics and emerging complexities of fast-changing accounts.”

Sagely described.

Clear thinking is difficult. Few of us begin by checking our premises or, sadly, our facts. Even in the domain of the concrete and verifiable factual information, so much rides on our implicit opinion of what exactly, in terms of data points, constitutes a ” fact” that we are usually off-base before we begin. Even if we are cognizant of these variables from the inception of forming a question, we might be horrified to discover, with some dogged investigation of the finer details, how fuzzy at the margins that even our peer-reviewed, “valid and reliable”, facts can be – much less the breezy assertions delivered by the MSM.

Then, more to the point of the KI post, there is the hasty selection of particular, reductionist analytical tools that a priori blind us to the nature of the emergent unknown that we are trying to understand. We become prisoners of our chosen perspective. One problem with human perception is that there is no guarantee, having recognized the existence of a novel dynamic phenomena, that our perception represents the most significant aspect of it. Much like conceptualizing an Elephant in motion from blind contact with it’s eyelashes. Or it’s feces.

Human nature is a perpetual rush to judgment. We must rise above that.

ADDED TO THE BOOKPILE

ANALYSIS IS A TWO-EDGED SWORD

The other day, Dan of tdaxp objected to my appreciation of logic as an analytical tool in the comment section:

“Humans are pretty terrible logical reasoners, so relying on logic doesn’t do much. Expertise comes from analogical reasoning, which allows faster processing, more reliable rejection of bad options, and effortless cognition in general.”

I agree with much of Dan’s statement. Most people are highly illogical and emotional in their decision-making process and not a few, though still a minority of the population, remain concrete thinkers their whole lives. We could not get anything done without cognitive automaticity kicking in to gear; having to consciously, sequentially, reason out the steps of every action would leave us exhausted before breakfast. My only quibble with Dan would be to replace “expertise” with “insight” or “creativity” as many variables, including the effects of chance, go into accumulating an experential base of knowledge. That is a minor divergence, for like Dan, I am an advocate of the benefits of synthesis, analogies, metaphors, horizontal thinking, metacognitive reflection and intuitive cognition – those forms of thinking that are generative of new insights.

I have also seen many complaints recently in the blogosphere about the negative effects of analytical thinking, if overused or abused, in these cases by lawyers, the profession most noted for elevating analytical reasoning above the factual variables to which the reasoning is to be applied. Here’s one:

From Kent’s Imperative:

“Worse yet, we detect a discernable strain of legal thinking which now seeks to impose restrictions not only on the collection of information, but on its use. The idea that a warrant might be required to search against previously accumulated foreign intelligence materials sounds absurd, but recent legal opinions appear to have laid the groundwork for such an argument in future cases. This would also be very nearly absolutely fatal in the context of fusion and collaboration for homeland security intelligence purposes (particularly if critical elements of the intelligence picture are obtained from foreign intelligence activities of DOD and other agencies, as if often the case.)

We have long maintained that the mindsets of the lawyer and the intelligence professional are diametrically opposed. The first seeks to present a structured picture through adversarial argumentation, and by training attacks to exclude evidence from the picture to support a particular viewpoint. The latter struggles to understand puzzles and mysteries, and to assemble a coherent narrative in the face of incomplete, conflicting, and deceptive information in order to support the decision-maker’s choices regarding courses of action. Allowing the lawyers to dictate further the key aspects of the world of intelligence – and allowing intelligence activities to be framed into an “investigative” basis rather than continuing inquiry into matters of standing interest – will be the death of the profession.”

Analytical thinking is a tool for reductionism. It helps you take the watch apart and explains how the gears fit together. It can identify broken parts and quantify performance. Analytical thinking works well in this instance because a mechanical watch represents a closed system with limited boundries. What analytical thinking can’t do, being an interruptive, destructive, activity is conceptualize an alternative way to tell time or arrive at Einstein’s insights about Relativity. For that you require a constructive, generative, pattern of thinking.

Used incorrectly, and analytical thinking often is because more people can analyze than synthesize well, it can destroy group dynamics and productivity through ” paralysis by analysis”. The downsides of all options become the most significant variables and opportunities are lost. Used correctly, to fine-tune, trouble-shoot, tweak and better adapt, analytical thinking can prevent disasters.

BRIEF MUSINGS

I’m preparing to leave town on another trip and find myself overstretched in terms of time but I have to note that Kent’s Imperative had some intriguing posts up ( hat tip to Michael Tanji) , about which I’d like to offer a few comments:

“Life at Google from an outside perspective“

Aside from seeing how uber-techies live and making me nostalgic about past years of reading defector-dissident Soviet bloc lit, I’d like to highlight this passage regarding a KI suggestion to the IC for personnel reform:

“A chance for line level workers to do the kind of intel they want to do (versus the latest crisis they have been thrown into), at least part of the time? Or to contribute to the literature of intelligence? (Modeled along Google’s 20% time.)”

My unqualified guess is that this would increase the productivity and prescience of the IC by roughly the same proportion that expanding private farming helped the Chinese economy under Deng Xiaoping. People typically generate their most valuable insights about those subjects which they are both curious as well as passionate – i.e. earlier in the learning curve than the status of graybeard authority ( once you think you know everything, you tend to stop learning).

The bar to doing this is not a manpower shortage but a middle-management fear of subordinate autonomy. Forcing a talented subordinate to do irrelevant busywork confirms a manager’s authority and power. Autonomous subordinates who do self-directed productive work tend to confirm the irrelevance of middle-management. Few managers have the psychological wherewithal to be adept facilitators, mentors or coaches of gifted employees as an efficient “management” outlook is an inimical perspective to generating creativity and sustaining ” unproductive” exploration.

“Regional versus functional issue accounts“

From a historian’s perspective, a cool post ( perhaps less interesting to others). Some historiography, lots of methodology. Money quote/conclusion:

“As for our opinions on the great divide between the two kinds of houses, we find ourselves veterans of uniquely transnational issues, having been subject to every manner of surge and task force and working group and crisis cell, in the most unusual of niches. We prefer to see small, aggressive, ad-hoc structures comprised of both analysts and operators from a wide range of issues and regional desks with interests and equities in the same target which overlaps their accounts. Only then, by throwing everything against the wall in a structure short lived enough to avoid its own bureaucracy, and disconnected enough to be (at least partially) immune from the day to day politics within a given agency or office, have we found the kind of answers we sought regarding the great questions of process.

We strongly believe such radically unstable and short lived environments are most effective because they are the very manifestation of Schumpeter’s process of creative destruction. It is certainly no way to create a sinecure, nor even to build a long term career path – but it is the best way we have found to generate new and innovative approaches and answers to hard target problems, and to the problems others have not yet begun to identify let alone address.”

Hear, Hear! Very strong agreement in a John Arqilla-esque vein.

It will happen but not until after several more disasters force that kind of transformation or an unusually bold and subtle visionary implements it on the quiet. There is far too much bureaucratic inertia because the vested interests prefer paralysis in which they hold the reins to successful action where they become recognized for the marginalized support staff they really are.

In my turn, if any KI gents happen upon this post, I suggest they look here. From this acorn of an idea, an oak will grow. Mark my words.

INTELLIGENCE, PUZZLES AND MYSTERIES

Gregory F. Treverton writing in the latest issue of The Smithsonian:

“Risks and Riddles“

“During the cold war, much of the job of U.S. intelligence was puzzle-solving—seeking answers to questions that had answers, even if we didn’t know them. How many missiles did the Soviet Union have? Where were they located? How far could they travel? How accurate were they? It made sense to approach the military strength of the Soviet Union as a puzzle—the sum of its units and weapons, and their quality. But the collapse of the Soviet Union and the rise of terrorism changed all that. Those events upended U.S. intelligence, to the point that its major challenge now is to frame mysteries….

….Puzzle-solving is frustrated by a lack of information. Given Washington’s need to find out how many warheads Moscow’s missiles carried, the United States spent billions of dollars on satellites and other data-collection systems. But puzzles are relatively stable. If a critical piece is missing one day, it usually remains valuable the next.

By contrast, mysteries often grow out of too much information. Until the 9/11 hijackers actually boarded their airplanes, their plan was a mystery, the clues to which were buried in too much “noise”—too many threat scenarios. So warnings from FBI agents in Minneapolis and Phoenix went unexplored. The hijackers were able to hide in plain sight. After the attacks, they became a puzzle: it was easy to pick up their trail. Solving puzzles is useful for detection. But framing mysteries is necessary for prevention. “

This article, though written for a general audience, struck a number of chords with me. Specifically:

* “Noise” is an important consideration in an era of attention scarcity economies. Eliciting a surge in ” white noise” by unrelated third parties ( say disinformation that sends pro-lifers off on a media campaign and in turn, energizers their pro-choice enemies to respond, diverting the attention of the general public to “X” degree) is useful camoflague. Purpleslog had a deservedly well-received post at Dreaming 5GW on ” the Puppetmaster” as a “5GW Archetype”. Such a mentality would cultivate media noise the way the KGB once set up and subsidized endless Communist front groups in the West.

* Uncertainty is relative. Some “mysteries” are more decipherable with a change of perspective, scale or temporal framework; others represent questions of deep uncertainty. Imaginative scenario planning exercises can help pattern recognizers familiarize themselves with latent possibilities ( Neo–Eurasianism ? Pan-Turanism ? A derivatives-driven implosion of globalization? Eco-extremist bioterrorists longing for planetary genocide?).

We need radical thought experimentation.

UPDATE:

IT security expert Gunnar Peterson has already covered this base well but from a different angle:

“Vulnerability Puzzles and Mysterious Threats“

“Risk differs from uncertainty in that risk may be measured and managed whereas uncertainty may not. Risk management efforts hinge on this important distinction because it highlights differences where a team may be more proactive. For instance, many vulnerabilities are known, hence they may be measured and managed whereas the threats to a systems contain a greater degree of uncertainty in that the threat environment contains numerous elements such as threat actors that one’s organization can not directly control.”