Human Transit

Transit projects, like all government projects, have costs and benefits, so one classic way to evaluate project proposals has been the "Cost/Benefit Analysis" (CBA).  Add up all the benefits, add up all the costs, divide benefits by costs and announce the Benefit/Cost Ratio (BCR).  If the it's below 1, which means the costs exceed the benefits, you kill the project.  The higher the it is, the better the project.

In the Financial Post (part of Canada's National Post), Peter Shawn Taylor argues that the Cost/Benefit Analysis is the best way to evaluate a project, and that bad projects are being advanced by a newfangled thing called Multiple Account Evaluation (MAE).  I encourage you to be bored by this dispute, but not before you understand it.  Neither of these methods is sufficient to end a passionate argument, and it's important to know why they never will be.

The problem with Cost/Benefit analysis is that it requires you to convert all the costs, and all the benefits, to the same currency.  That means you must know, with imperial confidence, the cost in dollars of such things as:

• each minute of each customer's time
• a particular ecosystem to be destroyed or preserved, which may involve various degrees of endangerment (of species, and of ecosystem types)
• historic or cultural resources to be destroyed or relocated, or preserved.
• the redevelopment potential of a particular area with or without the project, and the various benefits and costs arising from that potential.
• impacts of the project on affordability, and thus on the future shift of disadvantaged persons from one area to another, with a range of social impacts.
• benefits of electrification (quiet, no on-site pollution) on a neighborhood's quality of life, which impacts the previous point.
• a particular aesthetic impact that makes the city distinctive in a new way, such as a stunning piece of architecture or a new relationship to a unique historic artefact or feature of landscape.

Several of these items require confident prediction of the future, and there is also a vast question of who pays these costs and reaps these benefits.  I am not criticizing the vast body of research that has gone into improving the conversion factors that turn all costs and benefits into a dollar value.  But unless we really agree on what endangered ecosystems or municipal self-esteem are worth, in dollars, it's reasonable to question whether Cost/Benefit analysis can deliver the last, decisive word on whether a project deserves public investment.

One Canadian solution is the Multiple Account Evaluation (MAE).  Different types of benefits and costs are calculated differently and not converted into a common currency.  As Taylor describes it:

Developed by the [British Columbia] government in 1993 and now in widespread use, MAE dispenses with a single spreadsheet of advantages and disadvantages and adopts instead numerous separate “accounts:” a financial account, social account, environmental account and so on. In this way, the actual monetary costs and benefits of a project become just one of many issues to be considered.

A similar idea is inherent in the term "triple bottom line," which refers to economic, environmental, and social impacts — positive and negative — of a proposal.  In either case, you end up with a series of parallel analyses that give different answers from the point of view of different kinds of cost and benefit. 

So then what do you do?  Taylor quotes emeritus Professor John Shortreed of the University of Waterloo:

“The problem with MAE is that each account is given equal weight,” he observes. “This suggests the billion-dollar cost of the project is no more or less important than any of the other accounts, however trivial.”

It's easy to weight each account equally, because that sounds fair, but of course equal weighting is just as arbitrary as any other weighting, because we're talking about things that are not directly comparable to each other, such as a social cost vs an environmental one.  If you could really compare those things, we'd be able to do cost/benefit analysis.

MAE and the "triple bottom line" are useful concepts because they reveal the arbitrariness of weighting.  Weighting implies decisions, for example, about the relative importance of social vs environmental impacts.  That means the weighting is a value judgment about what matters to us as humans, as a community, as a civilization.  Surely we should argue passionately, maybe even irrationally, about that! 

The real problem here is that in the interests of consensus, we tend to allow technical analysis to make important value judgments for us, which is to say, we want technical analysis to tell us who we are. 

When you hear the terms social benefit, environmental benefit, and economic benefit, which arouses the strongest positive feelings?  The answer is an important signal about your deeply-held values and world-view.  If a technical analyis is making that decision for you, are you sure it's the analysis you want to trust?

Cost/benefit analysis and Multiple Account Evaluation (or "triple bottom line") both conceal value judgments.  Cost/benefit analysis hides value judgments in the factors used to convert various costs and benefits into dollars.  MAE or "triple bottom line," by contrast, comes up with multiple ratios — social vs environmental, for example — and ends up having to weight them, which is where the judgment appears.  I prefer the MAE or "triple bottom line" only becuase it makes the arbitrariness of weighting more visible, and hence pushes the conversation about it closer to the public sphere. 

It's understandable that we don't want to compare a billion-dollar pricetag to unquantifiable but powerful benefits that people will weight differently.  But the whole idea of the triple bottom line is that (a) we have to make these comparisons and (b) there's no technical basis for an answer.  When I'm doing evaluation frameworks of any kind, I look for guidance on the weighting based on locally adopted goals, value statements, or public discussion.  If the goals haven't been articulated, I often suggest some public discussion about the weighting itself.  (Some evaluation processes do consult the public on weighting, but the question can sound too technical at that point so not enough people pay attention; I think more can be done to make these discussions more vivid and consequential.)

As usual, I don't have the answer, only a refinement of the question.  Should communities talk about how to weigh competing values that are in conflict?  Or should they let those decisions be made inside a technical process in the guise of analysis?  There's a very powerful argument for the latter: decisions get made.  It's a lot of work to educate a community enough that it can express its values and desires in a form that a project can implement, even before you impose the other value judgments attached to various funding sources.  I don't advocate either position entirely, but have argued the "discuss the values" position because it should be visible as a choice.  In many ways, this is the "democracy vs technocracy" debate that we're hearing right now in the context of the European Union.  It's a hard question, maybe too hard, but it's one of those great debates worth caring about.