Define all costs as benefits, and double them: The problem with economic impact studies

When companies or governments propose a big project, like a pipeline or the Olympics, they'll often commission something called an economic impact analysis.

That's a report, often from a private consulting firm, that tries to quantify all the benefits to the economy of a given project. For example, Kinder Morgan's economic impact analysis of the Trans Mountain pipeline project, from the Conference Board of Canada, says the project will create 802,000 "person-years" of employment. 

That's roughly 40 thousand jobs for 20 years, from one pipeline.

If that sounds like a strangely large number for one pipeline, then you're on the same page as many academic economists. Economic impact analyses, the way they're commonly done, tend to inflate the numbers to such a degree they interfere with sober public debate, says Trevor Tombe, an assistant professor of economics at the University of Calgary. 

There's a joke among economists who look at economic impact studies, and we say "okay, define all the costs as benefits and then double them."- Trevor Tombe, University of Calgary

While Trevor doesn't oppose or support any particular project, he warns that claims like the one from Kinder Morgan's report are misleading.

In an interview with The 180's Jim Brown, Tombe says "you definitely want to be perceived as important economically in order to gain support for whatever it is you're putting forward. So pipelines want public support, and linking their contribution to the economy is one way to do it. For the Olympics, we see that prior to the Vancouver 2010 games there was an economic impact analysis that looked quite good. We're seeing that now in Calgary as well. Calgary saw an economic impact analysis for the Calgary 2026 games. And they often lead to very large numbers."

To help understand the numbers in these reports, like the 802,000 "person-years" of employment, Tombe breaks them down into three categories: direct, indirect, and induced. 

DIRECT

These are the jobs that come from the people actually building and working on the pipeline project. Those are the people hired by Kinder Morgan, with jobs like welding pipe and digging dirt and operating the actual pipeline. Compared to the claims of 802,000 jobs, Tombe says "that's going to be way smaller, only a few thousand person-years."

INDIRECT

This is where the numbers become questionable, according to Tombe. Indirect effects are the the jobs in other industries that come from the original project. For example, the people making the steel for Kinder Morgan's pipeline, and the people who make the machines that assemble the pipeline. That assumes those workers wouldn't be making stuff if it weren't for the pipeline. The economic technique that spits out these numbers is called an "input output model," and Tombe says the models "neglect where the money comes from, or where the workers come from, or where the machinery and capital come from. As far as the model is concerned, they all fall from the sky." In the example of Olympic spending, an input-output model neglects the fact that public money could be spent elsewhere, which would also create a different set of indirect employment effects.

INDUCED

This one is the greatest stretch for Tombe. "Induced effects" is an estimate of all the jobs created by people who now have money from their direct and indirect jobs, and are buying other things in the economy. So when the work crew on the pipeline project take a lunch break and head off to the deli, the server at the deli now gets counted into the benefits of the pipeline in the analysis. According to Tombe, the concept of "induced effects" treats money as if it will never be spent, if it weren't for this one project. For example, if Kinder Morgan spent its money on a different pipeline, there would be a different bunch of workers going to a different deli. Or if Kinder Morgan returned its money to shareholders instead of building a pipeline at all, those shareholders would also take that money and go for lunch.

Tombe says the results of these "input output models" provide such an inflated number, they interfere with debate in two ways. For one, they make it difficult to argue against building anything, when the promised numbers are so high. But they have another strange effect: turning important questions of what we value in society to one of mere numbers. 

"For the Olympics, this is less of a question about the economy. This is a question about the Olympics. Do we find them fun, national pride, civic pride, this sort of thing. And we should think of that decision more along the line of, is it a cost worth paying, rather than a kind of economic stimulus package, which I would be quite skeptical of."