When measuring velocity, “at rest” is defined by the observer’s frame of reference. The universe does not come with fixed points and fulcrums built in. Forget this, and it’s possible to get trapped in fruitless puzzlement over which of two objects in space is “really” moving and which standing still. Just such a memory lapse in this article from February’s issue of The American Prospect leads one writer to imagine that the universe of public opinion polling revolves around Chris Mooney.

I’m going to look at Chris’s treatment of a Zogby International poll on Social Security [PDF] commissioned by Cato, since that’s one I’m familiar with. Zogby found that when presented with this question:

There are some in government who advocate changing the Social Security system to give younger workers the choice to invest a portion of their Social Security taxes through individual accounts similar to IRAs or 401(k) plans. Would you [support or oppose]?

68% of voters were in favor. Foul! cries Mooney, because significantly fewer respond similarly if if you phrase the question in terms of “diverting” payroll taxes from Social Security, or if you raise the prospect of a “reduction in the guaranteed benefits retirees receive through the Social Security system.” For Mooney, this is proof that Zogby must be scamming the public to please his client by using loaded questions, rather than alternatives framed “more carefully” in order to bring out the “complexities of Social Security privatization.” These alternative ways of framing the question are presumably supposed to elicit more “authentic” or “informed” opinions.

If Mooney means to observe that the Zogby question leaves something out, he is, naturally, correct. But it’s hard to see how that’s to be avoided. After all, the reference to “reductions in benefits” might — indeed, almost certainly would — lead many people to think of cutting the checks current grandmas and grandpas are receiving, as opposed to merely reducing a planned growth in benefits for future retirees. You’d have to clarify that as well. But you still wouldn’t have a really “complete” or “balanced” question, some might say. People tend to automatically shy away from risk, so to really be evenhanded, don’t we have to mention the risk on the other side — that those “planned” benefits won’t actually be payable because the current PAY-Go system faces a fiscal crisis starting with a shortfall in payroll tax revenues starting in 2017? Don’t you also have to mention the possibility of higher total benefits even if guaranteed benefits are “reduced?” Should the pollsters correct misconceptions people might have about long term rates of return on the market? Is it important to emphasize that private accounts would be invested in a diversified portfolio of stocks, bonds, and other financial instruments, rather than handed directly to Ken Lay? Should respondents be reminded of the possibility of bequeathing their accounts, or even using them as collateral?

Doubtless opponents of privatiz… [cough], ahem, personal accounts could come up with their own list of countervailing considerations. That’s the point. A debate over which factors are “important enough” to include in this kind of question just is the debate over Social Security reform. Whether you’re for or against private accounts is precisely a function of which set of factors loom largest in your deliberation — a function of what you consider most important.

That’s not to say that if you put some work into it, and perhaps constructed a rather longer and more unwieldy question, you couldn’t come up with something most partisans of both camps could agree sounded balanced or “objective.” It’s certainly not to say that Zogby’s question is “right” and Mooney’s is “wrong.” That’s because a poll won’t tell you what “the public” “thinks” about “Social Security reform” — most people probably don’t think anything terribly determinate in advance of the question — it will only answer the question: what response are people prepared to give to this question right now? Articles like Mooney’s arise from the notion that there’s this thing, “public opinion,” that your questions are going to measure, and that it exists in a given magnitude independently of the quesition. Then we can fight about whether it’s been measured “correctly” or “incorrectly.” But the premise is almost certainly wrong.

Consider a familiar example from psychology. You have a group of sick people: with one treatment, exactly one-third will be saved, while with a second treatment, there’s a one-third chance all will be saved, and a two-thirds chance none will. People overwhelmingly chose the first option. But ask the same question, except framing the first option as one in which exactly two-thirds die, and the percentages flip — the respondents go from strongly risk averse to strongly risk loving. The Social Security debate is largely fought in the perception gap between those responses.

(Tangent: even with psychology out of the equation, Arrow’s Theorem has taught us that any system of deriving a social ordering (or a sample group ranking) over a range of options is going to exhibit at least one of five forms of intuitively unacceptable “irrationality” in some cases. It depends which options are presented, and in what order, and whether people are given raw pairwise comparisons, or a list to rank, or some number of “points” indicating desirability to assign to different alternatives. But I digress.)

Polls are still useful, since they can tell you which underlying sets of values (choice, security, wealth, family, symbolic unity) can be “tweaked” to produce one response or another. To demand more than this, though, is to ask the impossible. The Zogby results are true: they tell us how voters responded to the question asked. The other polls are also true: they tell us how voters respond to a different question. That said, Zogby’s question strikes me as a perfectly good one for its purpose — which is to say, it seems like a question worth gauging the response to. It tells reform advocates that they’ve got good prospects for a first move — a decent opening gambit. Mooney’s preferred question suggests a response that opponents of private accounts appear to have picked up on a while back: raise the prospect of throwing sweet old grandmothers into absolute destitution. Or, if you want to be a little more low key, do that thing where you redescribe reductions in proposed (unfunded) rates of increase as “benefit cuts for retirees.” But that’s not the last move either, of course. At each stage of the game — because at different points in the public debate, the “same” poll question will be charged with a different meaning — polls allow you to look ahead a turn or two. They are not an instrument for measuring a quantity — “approval” — of some fully considered outcome; they are device of representation for finding good moves in a discursive game. No poll can really call the outcome of public debate, since to do so, it would have to contain within itself the future course of that debate. As Hayek said of the mind, punditry cannot foresee its own advance.