Qualitative vs. Quantitative
Drew Conway of Zero Intelligence Agents asks a great question of all social science and humanities folk in the readership:
Nye and Drezner on Quantitative Scholarship
As a student in a department that covets rational choice and high-tech quantitative methods, I can assure you none of my training was dedicated to learning the classics of political science philosophy. On the other hand, what is stressed here-and in many other “quant departments”-is the importance of research design. This training requires a deep appreciation of qualitative work. If we are producing relevant work, we must ask ourselves: “How does this model/analysis apply to reality? What is the story I am telling with this model/analysis?”
Whether you are a producer, consumer or tourist of political science research you probably have an opinion on this debate, and I’d like to hear it.
Drew asks an important question. “Research Design” is inherently an act of qualitative and normative judgments. If the researcher is lacking a consciously constructed and identifiable intellectual framework or lens, they will still have one by default, except it is likely to be composed of contradictory hodgepodge of unconsciously acquired biases, hiding under a presumption of objectivity. That’s not an optimum perspective from which to select objects to measure and yardsticks with which to measure them.
The comment I left at Drew’s site was:
Quantitative analysis is sharpening the focus of the telescope or microscope. Qualitative analysis is knowing what’s worth looking at.
Being trained as a historian, I’m a qual dude but quant tools can tell me when I’m on target or by how much I may be off. Or if I am full of crap. On the other hand, quant scholars can be like drunks looking for their car keys under a streetlamp because that is where the light is. Quants need data and not every significant variable is the one that is easiest to isolate and measure. Or measure beyond mere correlation. Or at all.
Quant-Qual can never be either/or any more than we should try walking on one leg.
We need more consilience
May 23rd, 2009 at 12:21 pm
I find the idea that today’s political scientists eschew the classics of political science completely rather sad. It suggests to me that they are neither fish nor fowl.
.
I have no idea whatever of how today’s political scientists are trained but I have a pretty good idea of how the senior guys in the field, my contemporaries, were trained. My own undergraduate training included twelve quarters of math (at my university a calendar year was divided into four quarters and an academic year consisted of three quarters) and I had another six quarters of math in grad school. That included calculus, advanced algebra and set theory, advanced geometry, a solid year of probability and statistics, some applied math, etc., etc. As a grad student I tutored poli sci grad students, psychology grad students, and sociology grad students in using the statistics packages available at the university computer center (this was decades before personal computers–I was virtually unique in having a dial-up connection to the university computer center from my apartment).
.
I was appalled at their lack of mathematical knowledge. Not a one had taken a math course since trig in high school. They’d taken mandatory fluff statistics courses offered by their departments that mostly consisted of learning terminology. Nothing about the underlying mathematics. They could pump numbers into a statistics package but they didn’t have a clue about what the numbers really meant or their limitations.
.
As I said I have no idea what things are like now but those are undoubtedly the guys who taught Dan Drezner.
.
I agree with you, Mark. Two legs are necessary and, sorrowfully, I’m not sure they even have one.
May 23rd, 2009 at 3:31 pm
"Qualitative analysis is knowing what’s worth looking at."
.
I recall reading Eric Voegelin some years ago, who was saying that political scientists were no longer capable of discerning "theoretical relevance", in other words, they could not figure out what was and what was not important. He blamed this on a lot of people, including Max Weber, who wanted to do value free social science — yet who, Voegelin said, masked the scope of the problem he had created because he worked on serious and worthwhile problems and did not actually do what he said he was doing! Others who came later were, sadly "no Max Weber" and could not walk that tightrope.
.
Much of what happens that is critically important in very important human activities — politics and warfare preemenintly — is not subject to mathematical analysis.
.
Those who try to impose a veneer of quantification on these fields, beyond what they will actually bear, are doomed to failure. Robert MacNamara’s career should be the minatory legend to all who would make serious decisions while enveloped in a cloud of bogus, seemingly rigorous numerical analysis.
May 23rd, 2009 at 4:05 pm
Hi Zen,
I left this little comment to buttress your comment over at ZIA
I concur with Zen.
I too come from the historian craft, where it was recently commented that historians were the ultimate “cold case” detectives, combining Sherlock Holmes analysis with Dr. Watson’s lab to solve the problem. Take that away, and we become a Wild Bill Hickock, shooting every suspect til we get our man.
One of the positives in coming to history after a long career in logistics is the understanding of how quantative relates to any undertaking one chooses.
May 23rd, 2009 at 4:38 pm
ZP,
Thanks for the cross-post! I have responded to your’s and Tom’s comments over at ZIA (wish we could somehow merge the discussion).
In response to Dan and Lex, your concern about hacked mathematical analysis is exactly the problem in political science that departments like NYU (and several others) are trying to address with their training philosophy. My own research attempts to apply these rigorous methods to the analysis of conflict, and I would welcome your comments and criticism.
Not all of my colleagues share my sentiment for looking to answer questions of immeiately real-world relevance; but rather, are interested in pushing the theoretical boundaries–and that is important. As Zen points out, it should never be either/or. Important work will always be a sythesis of these perspectives.
May 23rd, 2009 at 8:45 pm
William Riker was one of the political science luminaries that introduced math into political science in the 50s. However, his major works usually mixed a deep understanding of history in with his math and game theory. His student Bruce Bueno de Mesquita writes in much the same vein. The quantity vs. quality dichotomy seems to come down to political science not producing well-rounded specialists (similar to economics).
May 23rd, 2009 at 10:05 pm
Hey gents,
.
Wonderful comments!
.
Dave – I’m appalled by my own lack of mathematical knowledge at times but I know enough to at least recognize the contours of my deficits and where to look to get assistance in remediating them. I can’t always say the same for other history ppl and I’m sometimes shocked by the scientific/mathematical illiteracy/innumeracy that prevails in discussions. Or in understanding how data relates to real world implications. Sort of the reverse image of the "no qual" problem in polisci.
.
HG99 – ah, logistics is a great complement to history – especially military history. The systemic perspective and interrelationships help us understand the possible. I like the "cold case" descriptor too.
.
Lex – I had not heard that criticism of Weber before but it sounds reasonable. I wonder what Seidlitz89 would have to say? Agree on McNamara.
.
Drew- I am not really well versed in what is happening in Polisci these days except via you, Duck of Minerva and a few other bloggers. It sounds like the field is in danger of walling itself in an intellectual cul-de-sac the way that philosophy did in the 1970’swith postmodernism and history did by going overboard on race and gender in the 80’s and early 90’s ( I still recall a guy I knew who did his dissertation on American financial speculation in the Manchurian railroads circa 1900, desperately trying to graft on a "feminist" angle in order to get it published even at a small, southern state university press).
.
josephfouche – I think you are right. Training should yield a mind with a primary set of analytical skills and a core of field expertise but also the capability for synthesis.
May 26th, 2009 at 12:13 am
One additional point, phrased as a question, to underscore the point I made above. Do today’s poli sci department chairmen, who by and large don’t have a great deal of math, really want to make poli sci a graduate-only field? That is, a field in which the prerequisites will require the equivalent of a full-fledged undergraduate degree? If they don’t, the political scientists they’re training won’t have the knowledge to approach their field soundly from a mathematical viewpoint.l If they do, they’re committing their departments to suicide.