zenpundit.com » big data

Archive for the ‘big data’ Category

The Automatic State?

Tuesday, October 29th, 2013

(by Adam Elkus. I will be guest posting occasionally at Zenpundit. I am a PhD student in Computational Social Science at George Mason University, and a blogger at The Fair Jilt, CTOVision, Analyst One, and my own blog of Rethinking Security. I write a column for War on the Rocks, and I once was a blogger at Abu Muquwama. You can follow me on Twitter here. )

I’ve been looking at some recurring themes regarding technocracy, control, elites, governance in debates surrounding the politics of algorithms, drone warfare, the Affordable Healthcare Act, and big data‘s implications for surveillance and privacy. Strangely enough, I thought of James Burnham.

Paleoconservative writer Burnham’s scribblings about the dawn of a “managerial revolution” gave rise to conservative fears about a “managerial state,” governed by a technocratic elite that utilizes bureaucracy for the purpose of social engineering. In Burnham’s original vision (which predicted capitalism would be replaced by socialism), the dominant elites were “managers” that controlled the means of production. But other conservative political thinkers later expanded this concept to refer to an abstract class of technocratic elites that ruled a large, bureaucratic system.

Burnham had a different vision of dystopia than George Orwell, who envisioned a rigid tyranny held together by regimentation, discipline, pervasive surveillance, and propaganda. Rather, the managerial state was an entity that structured choice. The conception of power that Burnham and others envisioned issued from dominance of the most important industrial production mechanisms, and the bureaucratic power of the modern state to subtly engineer cultural and political outcomes. Building on Burnham and those he influenced, one potential information-age extension of the “managerial” theory is the idea of the “automatic state.”

Automatic state is a loose term that collects various isolated ideas about a polity in which important regulatory powers are performed by computational agents of varying intelligence. These beliefs eschew the industrial-era horror of a High Modernist apocalypse of regimentation, division of labor, social engineering, and pervasive surveillance. The underlying architecture of the automatic state, though, is a product of specific political and cultural assumptions that influence design. Though assumed to be neutral, the system automatically, continuously, and pervasively implements regulations and decision rules that seek to shape, guide, and otherwise manipulate social behavior.

Indeed, a recurring theme in some important political and social debates underway is that changes in technology allow a small group of technocrats to control society by structuring choices. The data signatures that all individuals generate and the online networks they participate is a source of power for both the corporate and government worlds. The biases of algorithms is a topic of growing interest. Some explicitly link unprecedented ability to collect, analyze, and exploit data with enhanced forms of violence. Others argue that the ability to record and track large masses of data will prop up authoritarian governments.  Activists regard the drone itself–and the specter of autonomous weapons–as a robotic symbol of imperialism.

While an automatic state may be plausible elsewhere, the top-down implications of Burnham’s technocracy does not fit America fairly well. Some of the most prominent users of the relevant automatic state technologies are corporations. While cognitive delegation to some kind of machine intelligence can be seen in everything from machine learning systems  to airplane autopilot functions, it would be a big stretch to say that the powerful algorithms deployed in Silicon Valley and Fort Meade serve a macro-level social regulatory function.

Certainly it is clear that mastery of computational intelligence’s commercial applications has made a new Californian commercial elite, but it is mostly not interested in governance. Faulty government information technology deployment of large-scale systems (as seen in the Obamacare debacle) also does not auger well for an American automatic state elite. However, some interesting — and troubling — possibilities present themselves at state, country, and municipal levels of  governance.

Cash-strapped state governments seeking more precise ways of extracting tax revenue for road projects are seeking to put a mile-tracking black box in every car. Drivers would be charged based on a pay-per-mile system, and government planners hope that it can better incentivize certain driving patterns. Tools like the black box may suggest the dawn of a new age of revenue extraction enabled by cheap, precise, and persistent surveillance. Why not, say, utilize a black box which (in the manner of a traffic camera) automatically fines the driver for going over the speed limit or violating a traffic regulation?

In contrast to Burnham’s vision of technocratic elites, those who benefit from these technologies are the same unwieldy group of local bureaucrats that Americans must inevitably put up with every time they drudge down to their local DMV. While this may seem innocuous, local government’s thirst for new revenue has led to disturbing practices like the drug war habit of asset forfeiture. Though legal, asset forfeiture has stimulated corruption and also incentivized constant drug raiding in order to secure more funds.

What technologically-enhanced  local governments may bring is the specter of automatic and pervasive enforcement of law. The oft-stated notion that millions of Americans break at least several laws every day suggests why automatic and pervasive enforcement of rules and regulations could be problematic. As hinted in the previous reference to asset forfeiture, it is not merely a question of a rash reaction to substantial fiscal problems that local political elites face.

Politics is a game of incentives, and it is also a question of collective action and cooperation. As many people noted in analysis of mayoral corruption in the District of Columbia, many local politicians often have little hope of advancing to higher levels of prominence. Thus, they have much less incentive to delay gratification in the hope that a clean image will help them one day become more important. They can either reward themselves while they have power, or forfeit the potential gains of public office. Second, the relative autonomy of state and local governments is possible due to the lack of a top-down coordination mechanism seen in other, more statist political systems. The decision horizon, of, say, a county police department is extremely limited. So it will be expected to advocate for itself, regardless of the overall effect. These mechanisms are worsened by the fiscal impact of government dysfunction, the decay of infrastructure, privatization, and the limited resources increasingly available to state and local governments.

This mismatch is somewhat understandable, given the context of Burnham’s original theory. His inspiration was the then-dominant corporatist models seen in 1930s Germany, the Soviet Union, Italy, and other centrally planned industrial giants. He also misunderstood the political importance of the New Deal, claiming it was a sign of American transformation to a managerial state. As Lynn Dumenil noted in her history of interwar America (and her own lectures I attended as an undergrad), the New Deal was not a complete break from Herbert Hoover’s own conception of political economy. Hoover envisioned a form of corporatist planning in which the biggest corporate interests would harmoniously cooperate regarding the most important political-economic issues of the day,with the government as facilitator. The technocratic corporatism implied by Hoover’s vision was Burnham-like, and the New Deal was a continuation of this model. It differed only in that it made the government the driver of industrial political economy instead of designer and facilitator.

However, sustainment of a New Deal-like corporatist model depends on elite agreement. This was not to last. George Packer, Chris Hayes,  and Peter Turchin have all noted that today’s American elites do not have the level of cohesion necessary to sustain a technocratic state. Instead, they are competing with each other in a zero-sum manner. Silicon Valley entrepreneurs have flirted with the idea of secession. The US government cannot pass a budget that funds the government for more than a few months. A “submerged state” of  sub rosa government regulations twists policy towards an affluent few and private interests. The notion that financial regulation was compromised by regulatory capture is not controversial. Finally, a normative conception of elite appropriateness is no longer shared.

What this all suggests is that the impact of an automatic state will be scattered and aggregate. It will be experienced in large part locally through revenue-extracting technologies open up hitherto untapped sources of advantage. Political rent-seeking, not social engineering is the byword. The mechanism of extracting rents, however, is very “managerial” in its operation. In my home state of California, overt attempts to increase revenue have been consistently thwarted by political resistance. The potential for automatic state technologies to become “political technology” that fixes this problem through much less obvious revenue extraction mechanisms is understandably very attractive. However, the ability to process a constant stream of data from automatic state technologies will be contingent on computational power available, which will vary contextually.

Where the automatic state becomes politically and culturally influencing beyond pure rent extraction is also an area where localism will likely matter more. Computational capabilities for automatic enforcement and subtle structuring of political choice is difficult to accomplish on a national level except on a fairly piecemeal way due to national political constraints. However, on a local level where one party or interest may have vastly less constraining influences, it is much more likely that a computational instantiation designed to structure cultural or political choices toward a preferred result could occur. Even without such partisan considerations, there is always a school district that acts to ban a student’s behavior that they dislike or a prosecutor seeking to ramrod a given result that would see such technology as a boon.

All of this isn’t to completely dismiss the potential for federal usage of these technologies. But, as seen in the NSA scandal, mass domestic surveillance in an environment where the public is not afraid of a 9/11-esque event occurring may not be politically sustainable in its current form. A patchwork of “Little Brothers” tied to a revenue extraction mission, however, is a far more diffuse and difficult political issue to organize around.

If the automatic state comes, it is not likely that it will come in the form of a Big Brother-like figure hooked up to a giant machine. Rather, it might very well be a small black box in your car that measures your mileage–and is so successful that it is soon modified to track your speed and compliance with traffic regulations.

Blip: algo’s got rhythm at last!

Tuesday, August 27th, 2013

[ by Charles Cameron — a qualit with little time for quants making another graceful retraction ]
.

I haven’t been too convinced that algorithms were good at understanding my interests — remember that ad for “bold” Christian shirts (and babe) some fool code placed on Islamic Awakening — a site I was visiting to read up on Awlaqi?

Well, those algos are improving… Here’s what YouTube thinks I might want to listen to next, hot from the digital presses…

**

Turing Test: check!

I’d say YouTube’s algorithm has finally figured out — at least momentarily — the basics of who I am.

On two, one, seven plus or minus, and ten – towards infinity

Monday, July 29th, 2013

[ by Charles Cameron — a few quirky thoughts about graphs and analysis ]
.

Two eyes (heads, ideas, points of view) are better than one.

**

When I worked as senior analyst in a tiny think-shop, my boss would often ask me for an early indicator of some trend. My brain couldn’t handle that — I always needed two data points to see a pattern, and so I coined the mantra for myself, two is the first number. When the American Bankers Association during the Y2K scare wrote and posted a sermon to be delivered in synagogues, churches and mosques counseling trust in the banking system it was a curiosity. When the FBI, in response to the same Y2K scare, put out a manual for chiefs of police in which they provided input on the interpretation of the Book of Revelation, the two together became an indicator: they connected.

My human brain could see that at once — non-religious authority usurps theological function, times two.

For what it’s worth, the Starlight data-visualization system we used back then (1999) couldn’t put these two items together: I could and did.

**

To wax philosophical, in a manner asymptotic to bullshit:

One isn’t a number until there are two, because it’s limitless across all spectra and unique, and because it is its own, only context.

One isn’t a number unless there’s a mind to think of it — in which case it’s already an abstraction within that mind, and thus there are, minimally, two. At which point we are in the numbers game, and there may be many, many more than two — twenty, or plenty, or plenty-three, or the cube root of aleph null, or (ridiculous, I know) infinity-six…

Go, figure.

Two is the first number, because the two can mingle or separate, duel or duet: either way, there’s a connection, a link between them.

Links and connections are where meaning lies — in the edges of our graphs, where two nodes seamlessly integrate, much as two eyes or two ears give us stereoscopic vision or stereophonic sound, not by abstracting one from two by skipping the details that make a difference, but by incorporating the rich fullness of both to present a third which contains them fully via an added dimension of depth.

That’s the fundamental reason that DoubleQuotes are an ideal analytic tool for the human mind to work with: they’re the simplest form of graph — the dyad — populated with rich nodes and optimally rich associations between them.

**

**

Cornelius Castoriades wrote:

Philosophers almost always start by saying: “I want to see what being is, what reality is. Now, here is a table; what does this table show to me as characteristic of a real being?” No philosopher ever started by saying: “I want to see what being is, what reality is. Now, here is my memory of my dream of last night; what does this show me as characteristic of a real being?” No philosopher ever starts by saying “Let the Requiem of Mozart be a paradigm of being”, and seeing in the physical world a deficient mode of being, instead of looking at things the other way around, instead of seeing in the imaginary, i.e., human mode of existence, a deficient or secondary mode of being.

When I specified above “the simplest form of graph — the dyad — populated with rich nodes and optimally rich associations between them” I was offering a Castoriades-style reversal of approach, in which our choice of nodes is determined not by their abstraction — as single data points — but by their humanly intuited significance and rich complexity. Hence: anecdotes, quotes, emblems, graphics, snapshots, statistics — leaning to the qualitative side of things, but not omitting the quantitative. And their connection, intuited for the richness of the parallelisms and oppositions between them.

Often the first rich node will be present in the back of the mind — aviators wanting to learn how to fly a plane, but uninterested in how to land it — when the second falls into place — when a student asks a diving instructor to teach the diving technique, with no interest in learning to avoid the bends while coming back up. And bingo — the thing us understood, the pattern recognized, and an abstraction to “one way tasks” — including “one way tickets” established.

Let’s call that first node a “fly in the subconscious”. I’d love to have been a fly in the subconscious when SecDef Rumsfeld told a Town hall meeting in Baghdad, April 2003:

And unlike many armies in the world, you came not to conquer, not to occupy, but to liberate and the Iraqi people know this.

Because I could have chimed in cheerfully in the very British voice of General Sir Frederick Stanley Maude, in that different yet same Baghdad in 1917:

Our armies do not come into your cities and lands as conquerors or enemies, but as liberators…

Oh, the echo — the reverb!

**

The ideal number of nodes in the kind of graph I’m thinking of is found in terms of the human capacity to hold “seven plus or minus two” items in mind at the same time — thus, with a slight scanning of the eyes, a graph with eight to twelve nodes and twenty or so edges is about the limit of what can be comprehended.

The Kabbalistic Tree of Life, infinitely rich in meaning and instruction, has ten nodes and twenty-two edges. Once taken into the mature human mind, there is no end to it.

The value of a graph composed of such rich nodes and edges lies in the contemplation it affords our human minds and hearts.

**

Two, being the simplest number, will probably give you the richest graphs of all…

Art, in the person of Vincent Van Gogh, meet science, in the person of Theodore von Kármán.

What could have stopped Snowden

Thursday, June 13th, 2013

[ by Charles Cameron — mini-rant on importance of humans, human errors, and insight ]
.

OK, I watch TV & this screencap is from *Lie to Me* - this is about more than that


.

One proficient judge of human character with a good combo of micro-observation skills and / or gut instinct present at a Booz Allen job interview might very well have made a substantial difference, no?

**

I see this as a case to consider in terms of the human intelligence : number crunching ratio, or HUMINT : SIGINT balance.

Or rich data : Big Data or mind : machine question.

Or am I missing something?

**

Just one piece of the puzzle:

Microexpressions, from Wikipedia:

A microexpression is a brief, involuntary facial expression shown on the face of humans according to emotions experienced. They usually occur in high-stakes situations, where people have something to lose or gain. Microexpressions occur when a person is consciously trying to conceal all signs of how he or she is feeling, or when a person does not consciously know how he or she is feeling. Unlike regular facial expressions, it is difficult to hide microexpression reactions. Microexpressions express the seven universal emotions: disgust, anger, fear, sadness, happiness, surprise, and contempt. Nevertheless, in the 1990s, Paul Ekman expanded his list of basic emotions, including a range of positive and negative emotions not all of which are encoded in facial muscles. These emotions are amusement, contempt, embarrassment, excitement, guilt, pride, relief, satisfaction, pleasure, and shame. They are very brief in duration, lasting only 1/25 to 1/15 of a second.

Microwizards, from Wikipedia — which may not be quite the same as the ability to detect microexpressions:

O’Sullivan and Ekman identified only 50 people as Truth Wizards after testing 20,000 (~0.25%) from all walks of life, including the Secret Service, FBI, sheriffs, police, attorneys, arbitrators, psychologists, students, and many others. Surprisingly, while psychiatrists and law enforcement personnel showed no more aptitude than college freshmen, Secret Service agents were the most skilled.

**

That’s all, folks. It’s a beginning — what say you all?

Words, words — what’s a bunch of Wordsworth?

Sunday, May 19th, 2013

[ by Charles Cameron — bemused again, “jihad” (the word) in the news, “big data” too, plus Google expecting Mahdi ]
.

I suppose I should be glad — or should I? — that the word jihad is now in the news.

It’s about time. Jihad (the word, the concept, the interpretations) should have been in the news at least since 9/11, don’t you think? or since the World Trade Center bombing in February 26, 1993, perhaps? or at least since Osama bin Laden’s Declaration of Jihad Against the Americans Occupying the Land of the Two Holy Mosques of September 2, 1996?

In any case, the word finally seems to have arrived, if the entry from the National Geographic site last month (upper panel, above) can be trusted:

And Big Data (lower panel)?

President Obama launched his Big Data Initiative on March 29, 2012, but I’m not sure how long the term has been in active use. I’m told there’s no “big data” listing in the 2009 Oxford English Dictionary on CD-ROM, I have the sense that three days ago’s Foreign Policy is far more up to date than last month’s National Geographic in any case — and just a month or two ago the CTO of CIA, Ira “Gus” Hunt, situated Big Data somewhere between “the cloud” and right now, telling his audience at GigaOM:

Big Data was so last year, right, all those breathless articles and all the front page covers — I was expecting BD to be Time’s Man of the Year, right. This year what we’re really talking about is how do we get value out of the stuff?

That quote, of course, is so “two months ago”…

**

So what does National Geographic tell us about jihad?

The Boston marathon bombing has focused attention on the word “jihad.”

Vice President Biden characterized the alleged bombers as knockoff jihadis.” The Associated Press reported that the elder brother had “vaguely discussed jihad” with his mother over the phone in 2011.

Origins

“Jihad” is derived from the Arabic word juhd (meaning effort, exertion, or power) and literally translates to “struggle” or “resistance” for the sake of a goal. Used 30 times and in multiple contexts in the Koran, jihad most often denotes a struggle against external enemies, the devil, or one’s self. One example from the Koran (49.15) is: “The believers are those who believe in Allah and His Messenger … and jahadu (do jihad) with their properties and selves in the way of Allah.”

Mark Wilks, an early 19th-century British author, introduced jihad into the English lexicon, defining it as a Muslim “holy war,” in his Historical Sketches of the South of India. It’s retained that meaning in English; the Oxford English Dictionary defines jihad as “a religious war of Muslims against unbelievers.”

History

Because of its roots and context in the Koran, jihad has a positive meaning to Muslims. Whatever form jihad may take, the struggle is always noble. When the term is evoked against external enemies, it can be used only during just or defensive wars.

I’m sorry, but that last para beginning “Because of its roots and context in the Koran, jihad has a positive meaning to Muslims” isn’t terribly clear. When al-Zawahiri talks about jihad, for instance, does the writer imagine all Muslim readers imagine he’s talking about something noble? I fear there are some subtleties being missed her that not everyone who reads National Geographic may understand.

**

And what does Foreign Policy want to tell us about Big Data?

The promoters of big data would like us to believe that behind the lines of code and vast databases lie objective and universal insights into patterns of human behavior, be it consumer spending, criminal or terrorist acts, healthy habits, or employee productivity. But many big-data evangelists avoid taking a hard look at the weaknesses. Numbers can’t speak for themselves, and data sets — no matter their scale — are still objects of human design. The tools of big-data science, such as the Apache Hadoop software framework, do not immunize us from skews, gaps, and faulty assumptions. Those factors are particularly significant when big data tries to reflect the social world we live in, yet we can often be fooled into thinking that the results are somehow more objective than human opinions. Biases and blind spots exist in big data as much as they do in individual perceptions and experiences. Yet there is a problematic belief that bigger data is always better data and that correlation is as good as causation.

**

Mr Orange had something to say about the word “jihad” in his War Tracker blog the other day, under the title What’s in the names of terrorist groups (1): Jabhah al-Nusrah li-Ahl al-Shâm min Mujâhidî al-Shâm fi Sahât al Jihâd:

… they still use a religious term in their name: One that is quite negatively understood in the West but not so in the Arab and Muslim world namely Jihâd.

They are Mujâhidîn – those who do Jihâd (religious struggle – in this case fighting) – on the fields of Jihâd. Mujâhidîn has a positive, religiously legitimizing ring to it – see here is someone who struggles for the religion – and is furthermore including. Whether you are with the FSA (even one of the rather secular parts of that group mind you) or with an independent Islamist group or with Jabhah al-Nusrah all do use the term Mujâhîd and all may be identified by that term (Granted there was a time when Thuwâr (revolutionaries) was en vogue but no longer so).

That, IMO, gets us a lot closer to understanding a term that has a range of meanings, a range of users, and a range of audiences — from something along the lines of divinely obligated warfare to something akin to conscience (or what Rilke calls “being defeated, decisively, by constantly greater beings”) , and from those who use it for glorious self-identification to those for whom it is a euphemism for terrorist (irhabi), and from those itching for a fight to those longing, praying and working devotedly for peace…

**

So — since we’re talking big data and jihad, here’s a tiny snippet of jihad-related skew from Google, one of the giants of big data…

I came across it via SelfScholar, who posted a very interesting response re the Iranian nuclear fatwa issue here a few days ago, in a post titled Google Translate’s Khomeini Problem.

It appears that Google Translate has a distinctly unsecular view when it comes to major figures in Shi’ite theology — specifically, it adds religious honorifics to their names when translating from English into Farsi. As you might imagine, I wanted to know how they dealt with the Mahdi — and behold, my prayer was answered:

So Google awaits his blessed return?

It seems pretty clear that SelfScholar would be skeptical about that. He ends his blog post, in fact, with an indication that he neither awaits nor expects it — choosing for his final example “to highlight the inanity of it all, just one for the road…”:

**

Prof. Dr. Muhammad-Reza Fakhr-Rohani has an interesting piece titled Rendering Islamic Politeness Markers into English, which he concludes thus:

There remain some Desiderata to be dealt with. First, the Arabic pre-nominal honorifics as well as post-nominal honorific-cum-optative sentences must be re-translated with a view to remove the items which make the language sound odd, and perhaps ungrammatical. Secondly, appropriate abbreviations must be devised for them. Finally, they must find their ways into English dictionaries, hence registered as part of the language.


Switch to our mobile site