« This Week: A Tragic Legacy | Home | Spotlight on Fusion Voting: The Lost Tool of American Democracy »

Predicting with a Handicap: Why are Economists’ Predictions So Often Wrong?

user-pic

A provocative oped in today’s New York Times offers one answer to the question posed above: our understanding of incentives is too simplistic.

The piece in the Times got me thinking about this question of why members of my profession are competing with weathermen for accuracy kudos. So let me count the ways: here’s a list of reasons I think economists often get it wrong.

  1. Economists sometimes serve vested interests, and will change their views accordingly. The best example is also one of the best economists, Greg Mankiw. This textbook-writing Harvard prof was Bush’s chief economist for awhile, and during his confirmation hearing and subsequent tenure at the White House, he constantly defended Bushonomics, including supply-side beliefs that he once argued were the musings of “cranks and charlatans."

    Now, Mankiw may well have felt he could do the nation more good if he were working from the inside, trying to nudge the administration’s economic policy in a better direction (if so, he failed). But if we’re going to argue in support of ideas on Monday that we correctly dismissed as nutty last Friday, we’re unlikely to be either correct or credible.

  2. Economists are reductionists. For all the alleged complexities, much of economics is too simplistic to capture the myriad dynamics that drive economic outcomes. The minimum wage is a good example. The prediction that wage mandates above the wage set by the market should lead to extensive job loss should, theoretically, be a slam dunk. But the actual findings of the impact of an increase go from a little bit negative to a little bit positive. Even the more negative studies—the ones that find that mandated wage increases led to job losses—show that the benefits far outweigh the costs. And some very high quality studies show no losses at all.

    What gives? Apparently, the notion that an above-market wage leads to layoffs is too simplistic. I won’t elaborate the alternative ideas here: they include the idea that the market wage was sub-optimal in the first place, the notion that other mechanisms absorb the increase (higher productivity, lower profits, higher prices), and the fact that the world doesn’t work like the textbooks say it should.

  3. And one reason for that is, as the NYT oped argues, we misunderstand incentives. To be specific, we exaggerate them. Economists will predict large labor supply or investment responses to marginal tax cuts, and visa versa. That is, we’ll predict that people will work and invest a lot more if we lower the taxes on work and investment, and vice versa. In today’s version, you’ll hear vehement economic arguments that to bump tax rates up on tax hedge fund managers or to let the Bush tax cuts sunset will cripple the economy, as workers and investors respond by engaging in much less economic activity.

    Never happens. It’s not that people don’t respond in the predicted way. Some do, some don’t. But the responses never reach the advertised magnitudes.

  4. Old ideas die hard. For years, economists predicted that if the unemployment rate fell below six percent, inflation would not simply speed up, it would keep accelerating. Yet, in the 1990s, unemployment fell below four percent, and inflation decelerated. It turned out economists were playing from a rule book written for a time that had passed. Globalization, technology, and productivity gains had re-arranged the rule set that generated the old predictions, and we hadn’t caught up.

    Yet, many economists simply lowered their estimate of the unemployment floor—the rate you wouldn’t want to go below for fear of triggering an inflation outburst—to five percent. Well, we’ve been below that rate for a year and a half, and the relevant inflation gauge has once again been growing more slowly.

  5. Though we like to think otherwise, economics is not free of values, and that distorts our predictions. Many of the assumptions of classical economics are conservative by nature. Economists’ training teaches us to opt for market solutions and eschew government ones. Taxes, in economics, are assumed to generate “dead-weight losses” which certain doesn’t sound good (it’s the assumption that taxes leave all parties worse off than no taxes at all).

    We assume the unfettered market will almost always yield better outcomes, so we argue that a trade deal or a union agreement that mandates better labor standards, for example, will generate fewer benefits than one that lets the market rip. Yet, when these dire predictions fail to materialize—e.g., extensive research shows unions don’t hurt productivity growth—we fail to self-correct.

In practice, these problems interact with each other to compound the damage. Powerful vested interests, thinking incorrectly about incentives, predict that Health Savings Accounts (“consumer-driven” health care coverage) will be roundly embraced and cut costs. Anti-union ideologues bound to simplistic market models assume that injecting market competition into public schools will yield great results.

But in both of those cases, the predictions have been wrong. HSAs work for some people but they’re neither beloved, nor saving much (to the contrary, they’re causing new problems as people under-consume needed care). School voucher programs haven’t worked either, because the problem isn’t the lack of market competition, it’s the inequities in the economy leading to educational deficits that kids bring to school with them.

There are other things economists do well. Our empirical methods, in the right hands, can be highly informative and useful. But, like Yogi said, prediction is hard, especially when it comes to the future. When you’re carrying all this baggage along with you, it’s even harder.

62 Comments

| Leave a comment

I never bought the argument from my economics classes that raising the minimum wage leads to job loss. Sure, if the minimum wage was imposed on one company, that one company would be worse off. But, the minimum wage is enforced on all domestic companies at once. Customers don't just stop buying goods that they need from local companies, particularly if they have more money in their pockets since their wages just went up.

The thought that inflation would go up is countered by the fact that we now buy our goods from foreign countries that don't necessarily have a minimum wage themselves. The cost of goods aren't going up, not when someone overseas is getting paid a quarter an hour. Sure, they might have to work more since we have more money to spend, so their salary MAY go up - to thirty cents an hour. That kind of difference is not likely to cause a price difference from the product already on the self.

Poor people tend to spend all the money they earn. They don't hide it under the mattress like the wealthy do. Putting more money into the hands of more people makes the economy expand for everyone.

Economists strike me as generally a libertarian bunch. As such, they've let their values creep into their conclusions. In that sense, they see what they want to see. They've been blinded by their own politics.

Lots of food for thought here -- my favorite is the 5th reason economists often get their predictions wrong: that many of assumptions of classical economics are conservative by nature and to opt for market solutions over government ones -- and that most economists assume that the unfettered market will almost always yield better outcomes -- and any evidence to the contrary, they fail to self correct for.

Hartgal

How about this:

6) Economic is not a science.

Customers don't just stop buying goods that they need from local companies, particularly if they have more money in their pockets since their wages just went up.

The thought that inflation would go up is countered by the fact that we now buy our goods from foreign countries that don't necessarily have a minimum wage themselves.

These two statements are contradictory. If foreign goods are less expensive, consumers will indeed stop buying those goods from local companies.

Poor people tend to spend all the money they earn. They don't hide it under the mattress like the wealthy do.

Wealthy people do not hide their money under the mattress unless you consider investment in productive assets the mattress.

Deleted

Economists mostly suck at prediction and understandding of human behavior because they are not psychologists. Kahneman and Tversky wrote a whole series of papers in the 80s and 90s about the FRAMING of an economic proposition and it's effect on behavior. Prior to that, there was very little influence of psychology in economics. Admittedly, the behavior of capital markets often does not depend on individual psychological factors, but it does sometimes.

Good comments there. What we are learning now is that the private sector is inherently more corrupt, more bloated with morons and more full of lying scumbags than is the government. The absolute scandals in Iraq with Blackwater, KBR and Halliburton, in education, and in almost every area shows the inherent truth that government actors are more honest, better workers and more efficient than the bloated, corrupt, inefficient and criminally liable private sector.

Economists long for the day they are half as accurate as weathermen!

Good piece.

Seems to me what's hard to 'predict' is human behavior, yet it's always predicted as a part of any economic theory. (I must be missing something.)

Marx, Engels, whoever predicted that if man had cradle to grave security, he would be blissfully happy and a world under communism would be a Utopia. Wrong.

Adam Smith believed that innate in all men was a sense of 'compassion' which would come into play when the super rich - under laissez-faire capitalism there would be super rich - became aware of the abject poverty others were suffering, they would automatically share their largesse with same. Whoops.

I've always thought economists longed to be part of the science department - they will be when human behavior becomes a science.

"Wealthy people do not hide their money under the mattress unless you consider investment in productive assets the mattress."

He may be overstating things to equate investment with hiding money under a mattress. It however does not come close to providing stimulus to the economy equal to consumption.

If your capital markets are efficient, or even if they're not, there's a natural upper limit to the accuracy of publicly-disclosed macroeconomic forecasts. If my forecasts were reliable enough that I could consistently make money by taking the market positions that my forecasts suggested, I'd call my broker, not a press conference. If I announced them, everyone would rush in and my profit opportunity would be spoiled. I wouldn't even reap my due reputation, because all that arbitrage would knock my forecast off the mark.

The real question is, "do there exist hyperforecasters who can predict economic conditions well enough to prosper by acting upon them in secret?" My guess is, Yes.

[W]e misunderstand incentives. To be specific, we exaggerate them. Economists will predict large labor supply or investment responses to marginal tax cuts, and visa versa...

[T]he responses never reach the advertised magnitudes.

This is completely baffling to me. In the sciences I'm familiar with, when you keep finding your predictions are too high, you adjust your predictions.

There's plenty of economic data out there. Sure there are plenty of effects, and good model-building to get the major confounding effects in place (and isolate the remaining ones of interest) is tough. But isn't that model-building what economists get paid for?

Or, if the effect of an incentive is really so hard to discern under examination of the data, can it actually be that large?

I'd give you a gold star for this article, but I'm afraid the negative results of such a reward will discourage you from more efforts like this in the future!

People like Mankiw are an interesting case. I don't think he so much changed his views to get a job in the administration as that his views have changed over time as he has become more famous and powerful. He has become more authoritarian and a supporter of the powerful and the status quo. On his blog he deletes comments from those he strongly disagrees with. This is a sign of the "right wing authoritarian" personality type described by psychologist Robert Altemeyer.

For those interested here's a link to Altemeyer's free, online book describing this type of person:

The Authoritarians

And as is hinted at above, psychology plays a big, but under acknowledged, part in economic thought.

--- Policies not Politics
Daily Landscape

Interestingly, some economists are longing to be part of the psychology dept.  So-called behavioral economics (BE) is getting more attention these days. 

Broadly speaking, it relaxes the pervasive, underlying assupmtion that humans are rational economic actors, responding to incentives in predictable ways.

I think BE is a real advance, but one could easily say "whatever took you so long to get there!?"  You must appreciate, however, that it wasn't that long ago that economists recognized that some of our models didn't work because of "assymetric information," ie, the used car dealer was able to rip you off because he knew more than you about the junker he just sold you.

Excellent points and questions, professor.

I think the reason we don't self-correct enough is an intersection of problem 1 in the post (vested interests have their bejeweled thumbs on the scale), and yes, the impacts are very hard to tease out the data.

This latter problem is a function of insufficient data--many moving parts make it difficult to isolate incentive effects--and the actual impacts are much smaller than the theory would suggest. 

So it's like pulling invisible needles out of a haystack.

EG, textbook theory suggests large job loss responses to minimum wage increases or sizeable labor supply reactions to tax changes.  But we see neither. 

Yet, read any oped page and you'll soon stumble on some warning that tax increases, regardless of their magnitude, will lay the economy low, or tax cuts will lead to stellar growth rates.

I'm a psychologist with a BA and part of a postdoc in sociology. I'm a history buff and a long time research collaborator with anthropologists. Of all the social sciences, the one that has never made much sense to me has been economics. The basic assumptions--that money supply drives behavior and that decision making is utilitarian never made much sense to me. Economies exists without money and there are many ways to provide great "elasticity" to the economy through services, organized crime, etc. Economists have suddenly discovered that decision making is not utilitarian and that simple economic incentives don't drive behavior---essentially they have recognized the work of psychologists that was largely established 30 year ago. Daniel Kahnemann, one of the key players received the Econ Nobel a few years ago.

A simple understanding of society and psychology makes it obvious how economic forces fall apart. People at different income levels have diffe4rent strategies for using money, if only because they have different options. Market often appear competitive but usually seem to be distorted by virtue of cartels or monopolies that constrain supplies and supply chains and the influences of financiers (think Wall Street analysts who ruin businesses with strategies for short term gain that have negative long-term consequences like trimming staffs and selling real estate).

The practicing economists I know have been in the policy and public health spheres. Despite having their feet in the real world, they often seem constrained by orthodoxy and get angry because people and organizations don't really inhabit their world.

Weather prediction has gotten better over time, but frankly, anyone who looks outside and can read a barometer does just as well as the weather man in the short run. Someone who can read a map and understands which way the wind blows can do almost as well in the long run. Economists can recognize when a recession is already well on its way, but they can't see how money markets and human behavior change over time. The effort to gain precision in prediction, as in the case of econometrics in the 70s and 80s was doomed because it has relied on simplistic assumptions that don't model actual behavior very well.

Mankiw has disappointed many, including somewhat contrarian economists like Max Sawicky. I think, like most Bushies, he has to tow the line or no one will listen to him. Of course, towing the line means that his influence on the administration is marginal at best. He's pretty much destroyed his academic career, although he may have opened up doors for himself for future prostitution in the wingnut sphere and in corporate America.

I would expect that the vested interest influence dominates. The economists most likely to be quoted in the press work for investment banks or advocacy organizations (notably the National Realtors Collective, or whatever it's called) and are incited to mislead the public into misallocating its resources.

The press fails to apply analysis and only seeks false balance, so we hear that economists believe these wrong, crazy things that of course don't come to pass (remember the NAFTA 'debate'?) and we think they're flawed. They're really just against us.

Your use of the word "classical," by analogy to physics, evokes another major failing of economists. They do not understand nonlinear models; they think of the world as an interlocking system of partial differential equations.

The old notion was "simple models, simple behaviors." If you picked up a complex behavior, you needed a complex model to explain it. It turns out that this is far from true. You can't blithely assume that the "invisible hand" will guide a system to a stable equilibrium, because in a lot of cases, it won't: there are a lot of simple models that, for certain ranges of parameter values, either oscillate periodically or just behave chaotically.

If something like fluid dynamics, with its simpler feedback mechanisms, requires this to be taken into account, how much more must a working economy, with all its informational feedback loops, require it?

Adherence to Adam Smith's concepts is a religious issue. There is no Marxist Utopia any more or less than there is a truly free market, and any mention "free market" or "utopia" should be a red exclamation point that any assertion following is speculative.

Religious issue? I'm fascinated. What religious issue? (But admittedly for the likes of Milton F. laissez-faire capitalism more closely resembles a religious experience than a mundane, workable economic theory.)

I'm not so sure a Marxist Utopia is a pipe dream. Russia after 30 years of two wars on her soil and a revolution to boot was a seriously flawed candidate to test any economic theory.

All said, any cure-all theory of anything must always require the raising of red exclamation points.

Not an economist, but I'd been given to understand that America is unique in its hubris to name the subject here 'Economics,' thereby obscuring the political underpinning of the version of economics being taught, as if political theory didn't enter into it.

The reality of socialist economics, or liberal (in the European sense) economics, etc., ie., politically-informed economics, is unknown to American students. The result is that most 'American Liberal' economics run counter to the classical theories in economics, which are 'American conservative' and therefore inherently antithetical to the economics most Americans actually endorse.

For instance, I do not think that most accounting 'externalities' should be ignored or, at best, corrected by the use of tax money. These include various pollutions, much poverty, most of our immigration 'problem,' as well as the epidemic of un-insurance for health. These are caused in large part by successful off-loading of the cost of process improvement or waste clean-up, of the cost of competitive wages for, or to attract, American workers, etc.

Yet American economics, exemplified by GAAP standards and American law, allow these externalities, which you and I pay for in so many ways, and which provide much of the profit and ridiculous compensation levels of the current corporate dukedoms in the US.

Legislation could require theses costs to be borne at more appropriate locations in the production/provision process. The technology to identify these costs already exists, and is largely in place in most manufacturing and service companies.

Economics is always political, and to imply that it is empirical in any but the most sterile experiments is laughable.

Speaking of laughable: A man walks into the library of the Economics department.
"Do you have any books with a happy ending?"

Most economists don't deal with facts, or even actual experiences. They have little to back up what turn out to be pet theories or, even worse, feelings. When someone actually does research, and looks at facts, there is a chance that an economist's predictions may have some accuracy. Otherwise, it's like the pundits on the Sunday Talk Programs.

I agree with the thrust of your comments and note that "political economy," a hybrid discipline that is much more closely linked to reality than what is today called "neo-classical economics" doesn't get much respect in academia.

For one, political economy elevates the role of power dynamics as a critical determinant of economic outcomes. 

Eh. What is a science anyway? I used to think it was defined by the scientific method, but a few grad school courses covering paradigms and epistemologies got me off that wagon. Even in the hardest "sciences," the disproving of hypotheses never really happens, and the statistics that stand in for it now are better read as persuasive text than immutable data.

Economics' real problem, in that regard, is that it has an enormous case of physics envy. It so desperately wants to find Newtonian laws from which it can axiomatically extrapolate predictions for the universe of human economic exchanges. While I could rant on about the lack of respect for valuable concepts in Keynes, Marx, Veblen, Weber, and a host of others that get routinely ignored in neoclassical economics, the main critique I would say of economics is that it just needs to get empirical. If you can't back up your theory with empirical examples, you shouldn't be published in high level journals.

Sadly, economists seem to regard empiricism as some sort of quaint trinket for more lower forms of study.

XopherMV: Customers don't just stop buying goods that they need from local companies, particularly if they have more money in their pockets since their wages just went up.

The thought that inflation would go up is countered by the fact that we now buy our goods from foreign countries that don't necessarily have a minimum wage themselves.

Robert Brown: These two statements are contradictory. If foreign goods are less expensive, consumers will indeed stop buying those goods from local companies.

What? That's not contradictory. Are consumers going to buy their groceries, clothes, electronics, and fast food overseas? No. They're going to go to their local store and buy them there. Any increase to the minimum wage won't effect that.

Local manufacturers may feel a pinch if they haven't already outsourced their production. The price of American-label, foreign-made goods won't go up much, if at all, as a result of the minimum wage increase - there's too much competition overseas. The main difference will be that Americans can afford more of those goods, with the bulk of the profits from increased demand going to American companies rather than the foreign manufacturers. If prices go up it will be because American companies want to raise their prices, not because they have to.

the main critique I would say of economics is that it just needs to get empirical. If you can't back up your theory with empirical examples, you shouldn't be published in high level journals.

I would say that is the essence of science. Sure, we can have a lot of philosophical and post-modernist discussions about the scientific method and meta-narratives or whatever, but when it comes down to it, economic hypotheses are rarely if tested against the world.

For what it's worth, even Kuhn didn't think that the products of the "hard" sciences were social constructs.

Exactly. Add on top the fact that the public sector isn't paying tens to hundreds of millions each year to CEOs and their executive teams. No one on the public dime comes close to that, which would be considered wasteful spending in their sector.

Add on top the fact that the public sector isn't paying out profit to millions of shareholders or buying back millions of shares of company stock.

Add on top the fact that the public sector can obtain huge economies of scale for virtually everything they do.

When government runs well, it can be an amazing thing. Look at Medicare compared to the HMOs. We would actually save money if we all went with Medicare, a government system.

Most people get economics wrong by misapplying micro thinking to macro problems. Thus, what Wal-Mart does is just like hiring little 12-year-old Jimmy next door to mow your lawn. Who are you to tell poor little Jimmy he can't cut grass for 10 bucks? Jimmy WANTS to cut your grass. He LOVES to ride your power mower.

Of course, Wal-Mart executives WANT people to confuse their behemoth corporation with little Jimmy's kindly neighbor. Vested interests facilitate all sorts of confusion.

The proper way to think about the minimum wage was laid out (if I remember right) by John Kenneth Galbraith. To paraphrase heavily, demand for Double Sloppyburgers is the price people will pay for one, which is a measure of how badly they want it.

If that market price is too low for the burger flipper to earn minimum wage,  that means people don't value Double Sloppyburgers very much. Therefore, who cares if that burger joint goes out of business. Nobody really liked their Sloppyburgers anyway.

6) Economic is not a science.

since scientists aren't omniscient, this one probably doesn't matter.

To boldly go...

How about we change this to:

6) Economics is not a science yet.

There is no reason economics cannot be a behavioral science. Economics could become a science if economists simply begin using tools already used by psychologists to experiment on human behavior. The reason it isn't a science is simply that it is not approached scientifically and empirically, with studies done to prove each hypothesis. But economics is not inherently an unscientific field. All that must be done, is that economists should be trained more in the scientific method. Meteorologists don't have enough data or computing power. Economists by and large refuse to scientifically analyze their own.

I agree completely that economics suffers from physics envy. As you and others have pointed out, economic theories are rarely used to make real world predictions, and those predictions are frequently well off the mark. Unlike physics, chemistry, biology, and other sciences, the economists don't look at their failures and go back to the drawing board.

If you follow science you realize that theories are disproved all the time. Wasn't there a theory that the genetic code carried all the information for developing a new individual? That ignores issues of cross generation gene activation. The genome is not enough. Wasn't there a theory that evolution described a tree of life, but inheritance at the single cell level allows cross species genetic transfers. Wasn't there a theory that refractive indices had to be positive? Now they are creating pseudo-materials with negative indices. Wasn't there a theory that all matter interacts with light the way familiar matter does? Now astronomers using gravitational lensing to find dark matter that doesn't. Wasn't there a theory that electron pairing limits the maximum superconducting temperature? That went by the boards in the 80s.

The trashcan is a scientist's best friend. It's high time economists started learning how to use one.

How to become a hyperforecaster:

1) Get 1024 clients.
2) Predict a bull market to 512 and a bear market to 512. Advise them accordingly.
3) Lose half of your customers.
4) Predict a bull market to 256 and a bear market to 256. Advise them accordingly.
5) Lose half of your customers
... and so on ...

After a certain point you will have a customer base that thinks you are god.

Yes, I know, this is an old joke, but it is better than "flucted again".

Actually, long term weather forecasts are much better than they were 10 or 20 years ago, and the weatherman does a much better job of forecasting than I can. Of course, they have spent years building, testing and throwing out models. The people having trouble are the river flow forecasters. I just read an article in which they realized that river flow forecasts haven't improved in 20 years, and unlike economists, the river forecasters feel that they need to do something about the problem. Economists don't even realize that they HAVE a problem.

I've always had a lot of respect for the New Dealers, and I've made good money on the market following their precepts. The simple fact is that the less money you make, the larger fraction you spend. Eventually. rich people get all the money, and the economy winds down with the poor just scraping by and a static power curve of wealth. This describes much of the third world today.

The problem with supply side economics is that rich people are not necessarily stupid. Why should they invest if no one has money to buy their products? If society, and people, are to get even richer, something has to pump a lot of money from the top down to the bottom again. Few rich people will do this voluntarily because it would place them at a disadvantage with respect to other rich people. Once again, saying rich people are stupid is poor basis for an economic theory or investment strategy.

The market generally goes up when:

- taxes are raised, and I'll consider the minimum wage a tax,

- government borrowing and spending increase, but a lot depends on what the government is wasting the money on,

- the economy tanks and there is no place to invest productively so people buy stocks.

It doesn't make much sense, but it works.

So rich people should be encouraged to buy private jets instead of building factories?

From an earlier post that seems to fit

When an economist is chanting the mantra "On the one hand, but on ...."
and raising and lowering each hand what is actual happening
is a display of free market auctions at work.

The economist is trying to bring forth a
higher bid by the opposing side
wanting their answer to be "The Answer".

If asked in private about his actions, the economist would say,
using the free market brought forth the best answer.


-----------------------------------------------
Today, are we searching for I deals or Ideals?
-Thinking

Bravo csp. A couple of days out meteorologists are very accurate. Economists can't find their fannies with their own hands for those reasons cited by so many above.

So often economists are so incompetent as to be laughable. Over the years one of the funniest guys was Greenspan.

The really scary part of the disfunctional 'discipline' is that it seems to be getting worse.