If you live in the Portland metro region (Oregon side only), the regional government Metro wants you to be part of an online panel that comments on key issues facing the region. It's called Opt-in, and you can read about it, and join it, here.
When I tweeted about this yesterday, several people I respect commented that Opt-In sometimes asks wildly distorted and leading questions. I can now verify that this is the case, based on a survey it asked me to take this morning which includes this appalling question (click to sharpen):
Two choices about transit and they're both about expanding rail. Bus infrastructure and fleet are not mentioned even though buses are still the dominant mode in Portland and Bus Rapid Transit is a serious option in two rapid transit corridors now under study. The designer of this question appears to be unfamiliar with Metro's own transit planning process, and is imposing their own biases on the survey. That may be understandable in the early stages of a tool's develoment, when Metro's leadership may not be paying attention yet, but it should be unacceptable and if the tool thrives, it soon will be.
But is this a reason not to join? No, it's the opposite. Biased questions are a reason to use the little text feedback boxes to complain about biased questions, and to contact your Metro councilor if you feel strongly about them. When Metro councilors will start getting letters about biased questions being asked in Metro's name, the problem will be righted quickly. Just as boycotting elections is a bad way to get your views represented in government, boycotting survey tools is a bad way to get them to ask better questions!
Like all self-appointed panels, OptIn reaches a distorted sample of the population, but so does the massively inefficient and exhausting ritual that it could someday replace: the public meeting. (The FAQs gently suggest this long-term prospect, though I'm sure we won't see the end of the public meeting soon.) Public meetings still have value when they are organized as genuine conversations, such as interactive workshops that I'm often hired to run. But if the purpose of a public meeting is for you to go and give a speech about why you're right and everyone else is wrong, well, we can all save carbon emissions and time out of our busy lives by doing that at home in our pyjamas.
Need to smash the patriarchy or abolish the government? I admit online surveys aren't very satisfying if you're massively angry and need something destroyed, and for that your options are still voting, peaceful demonstrations, and if necessary civil disobedience. But online surveys are a great way to hear from a vast array of voters who are willing to communicate more thoughtfully and with less effort, and who tend to be shouted down or intimidated in public events. Yes, there are issues about exclusion of low-resource groups and those with language and educational barriers, but Metro's surveys look pretty accessible to anyone with high-school literacy in English and they'd be easy enough to translate and supplement with spoken text.
Metro's currently asking its panel about its priorities for the future of transit in the region, and unlike the survey I mention above, it's a good and worthwhile survey that everyone should complete.
Again, I wish the transit survey had the courage to ask about new funding sources. The region's transit agency has cut over 15% of its service in the last decade, and is in ongong budget difficulty due to unfunded health and pension commitments made in wealthier times. While it may gradually restore a basic functional network, it is nowhere close to offering the pace of service growth that would support the region's land use vision or its justify its green reputation. But the questions the survey does ask, about your priorites for deploying what resources can be found, are mostly clear and reflect the realities of transit's geometry and costs.
If you don't enjoy public meetings, or have better things to do with your time, join Opt In, and if you're not in the Portland region, encourage your regional government to emulate it. And if you find a biased question, comment about it!
Interestingly, Toronto just finished doing many of Jarrett’s recommendations. After a half hour presentation, we were divided into discussion groups of about eight.
One ‘discussion guide’ we individually filled out was about funding sources. The second was ‘transportation planning’ which did not prioritize the various plans, but asked us about view transportation, land use and economic development.
They are Guides A & B at the bottom of:
Hopefully Jarrett can review ‘A’ and provide some comment.
While the questions are biased in the limit of ranges that are offered I bet that many of those who designed it do not believed it is biased. After all it is just questioning the relative importance of the OBVIOUS important items. Too bad if they don’t reflect your values.
At least they are not like the questions in push polls that ask questions like:
Do you agree with the governments plans to take your hard earned tax dollars currently earmarked for “Good Roads” and spend it on items like “Subsidized transport for the unemployed?”
Robert Wightman. Presuming a shared notion of the “OBVIOUS” is one of the most common manifestations of bias! I’m sure that one or a few well-intentioned people were doing their best with this list of options, but this is why survey design is a professional expertise and one that requires a lot of local listening to get right. Metro will need to raise its game here if this tool is to become credible. That’s all I’m saying!
Not to defend Metro at all–indeed, if true I consider this a somewhat damning observation–but the list appears to be capital-projects focused. Which might explain a lack of reference to local bus enhancements (though not BRT).
While Metro isn’t entirely clueless about operations, there may be many factors which keep ops out of its focus: most grant money (particularly from the Feds) is for capital projects, and planning agencies generally have had an unfortunate tendency to neglect things like ops and maintenance.
Jarrett. I was hoping to be facetious but perhaps I was too subtle. The reason that these people do not use professional designers is because they KNOW the OBVIOUS and don’t need to WASTE money paying someone else. I just watched a documentary on Robert Moses. He was a master at knowing what people REALLY WANTED, even if they didn’t.
I have a problem with these web surveys because it is so easy to push the reults one way or the other by multiple entries. A well designed poll is much better, but more costly.
I thought this would be about having “urgent” and “high” on one end but only “low” on the other, its not balanced.
I’m having trouble seeing the value of these sorts of surveys apart from a possible PR function to appease interest groups. A secret ballot on infrastructure priorities seems helpful, but nobody sitting at home in their pyjamas has the understanding needed to properly sort through these priorities if they have avoided hearing other people’s points of view. If both the questions and results are biased, and not representative of the broader community, what possible use could these surveys have in a responsible decision making process?
It’s pretty scary if decision makers need to rely on this sort of dodgy feedback to sort out important long term priorities for their cities. Have the citizens elected leaders who don’t know what to do, or how to prioritize?