Walkscore.com and the Lure of the Single “Score”

The Conservative Planner [blog site no longer active] has a thoughtful attack on WalkScore.com‘s methodology for calculating a simple “walkability score” for any neighborhood in America.  He’s found several examples where WalkScore has given a high score to a place that’s clearly hostile to pedestrians when viewed on the ground. 

One only had to take a stroll around the Hilton in downtown Atlanta to see the inherent flaws in this online tool. Type the Hilton’s address in Walk Score (255 Courtland Street NE, Atlanta, GA) and it will tell you that it is a “86 – Walker’s Paradise”.  [JW:  Actually, it now says “86 – Very Walkable’]


The Conservative Planner continues:

Nevermind the fact there are 4- and 5-lane one-way streets with high speed traffic surrounding the hotel. Interstates 85/75 runs to the north and east of the hotel and is a major pedestrian barrier. The restaurants that are deemed by Walk Score to be nearby and within walking distance are buried within the Marriott and Hilton hotels, which are 1960s/1970s behemoths. It is hardly a walking paradise; it may be in a downtown area but the street system is strikingly suburban.

Fair enough.  But here’s the Conservative Planner’s suggested remedy:

Walk Score should not be used to analyze how walkable an area is as it does not consider the three most critical factors of walkability:

1. Connections: Walk Score can’t tell if sidewalks exist or not. It can’t even evaluate if there’s a street to connect to a destination that it tells you is within a walking distance of your address. You might need a machete and steel-toed books to walk to the doctor’s office.

2. Actual Route: Walk Score measures crowflight distance, not actual walking distance. I live in a neighborhood that has several destinations within a 1/2-mile radius. Too bad it takes me more than a mile to walk to them because of the lack of a pedestrian system along a 7-lane arterial highway with no crosswalks and poor connectivity despite its location within a streetcar suburb.

3. Land Use/Design: You could have the most “walkable” area according to Walk Score, but if you’re walking in front of a Walmart or other horribly designed commercial or residential strip, you won’t find the area to be very pedestrian-friendly.

These are all valid points, but if the Conservative Planner is really a planner, he/she would know how hard this bar is to meet.  Items 1 and 2 require that every local government in America provide a digitized map of its entire pedestrian network, showing not just which streets have sidewalks but where the usable off-road paths are, possibly including parking lots and vacant lots that you can safely cut across.  It should probably also show not just pedestrian signals but information on how they’re timed.  Many local governments don’t have this information.  Some have it only in the form of hand drawings on paper maps lovingly curated by the town’s sole “pedestrian planner,” who is allowed to play with these things in his broom-closet office so long as he stays out of the way of the road engineers.

As for item 3, “design,” well, sure, most of us would rather walk through a park rather than across a parking lot, but “design” by its nature refers to a subjective response, not a scorable metric.  (I’d make an exception for Robert Cervero’s use of the word in The Transit Metropolis, which I suspect was selected more for alliteration in his phrase “density and design.”  By “design” Cervero really means network connectivity — the Conservative Planner’s #2 — measurable, say, by the percentage of a 1/4 mile air radius that’s within a 1/4 mile walk on the pedestrian network; more on this here.)

What’s more, even if we could agree on a scorable metric, you’d need a nationwide database of exactly how every lot has been developed, continuously updated of course.  The closest thing we have to that is Google Street View.  Is someone working on a computer algorithm that will study every Street View photo in the country and assign a universally-respected “design score”?  If so, perhaps we can look for improvement.

Bottom line:  What the Conservative Planner is really pointing out is that prevailing data structures are designed to the needs of the prevailing mode — and right now, that’s cars.  Yes, every local government should have a complete database of its pedestrian links, and yes, they should send it to Google so it can be added to Google Maps.  Only then will we have an automated measure of actual walking distance.

Meanwhile, the sensible response from WalkScore.com would be to move their “How WalkScore Doesn’t Work” page to a more prominent place on the site.  A sensible response from the rest of us would be to remember that any methodology that reduces diverse inputs to a single score is not just an approximation, but an approximation shot through with value judgments that the score’s consumer may not share.  WalkScore could help remind us of this by showing, on the front page where it displays the score, the separate “sub-scores” from which it’s calculated, so that we can each decide if the factors are weighed just as we’d like.

UPDATE:  Matt Lerner of WalkScore tweets that they are working on these problems!

15 Responses to Walkscore.com and the Lure of the Single “Score”

  1. M1EK October 25, 2010 at 10:39 am #

    Actually, all WalkScore would need to work much better than it does now is some crowdsourcing support. “Rate this street segment for walkability”, IE.

  2. Eric Fischer October 25, 2010 at 11:12 am #

    Walk Score has said that they are working on making it use real walking distances instead of crow-flies distances (http://blog.walkscore.com/2010/08/street-smart-walk-score/), although that still won’t take the characteristics of the streets into account.
    I hope somebody is figuring out how to rate the walkability of places from Street View pictures, but it’s a really hard problem because the characteristics of the good places can vary so widely yet the difference between good and bad can be very subtle. I think the cellular phone carriers probably actually know with pretty good accuracy what streets people walk on and which they avoid, but they aren’t telling.

  3. Steven Vance October 25, 2010 at 11:24 am #

    Would the criticism subside if WalkScore just chose a different adjective?
    It seems the criticism is based entirely on the word, “walkable.”
    If instead WalkScore said, “Your score is 86 – lots of businesses nearby,” I don’t think of any this would be relevant.
    Even if WalkScore doesn’t take into account what really makes a place walkable in its calculation, one main purpose for the score remains: how varied and how many businesses are within walking distance?

  4. PlebisPower October 25, 2010 at 1:02 pm #

    Great observations regarding the need to rethink how we measure and evaluate walkability and many other bases for making policy (transportation and otherwise). After all, men first calibrated length and distance in human terms to measure of the world in relation to himself – a system that clearly needed rethinking. Disclosing the WalkScore methodology and applicability more prominently is a way of acknowledging its inherent limitations, and so is a good idea too.
    But the suitability of the WalkScore formula is a red-herring; there is no algorithm that will apply in the real world, IMO.
    Instead I’d call attention to the observed disjuncture between the WalkScore evaluation and conditions (as observed by the Conservative Planner). If preconditions exist for a walkable community (mixed uses, nodes, etc.) but in practice the walkability of the place is attenuated or neutered as built, it seems to me that the WalkScore can be a useful metric or benchmark to evaluate sub-optimal outcomes.
    So I’m less interested in a single score because it’s never very useful. Instead I’d like a rubric. Then we can benchmark potential as a way of looking across them for lessons in what otherwise would be an unwieldy, messy comparison given the real world.

  5. Leigh Holcombe October 25, 2010 at 1:58 pm #

    When I first tried WalkScore and tried to rate my neighborhood, the greatest flaw I noticed was the ignorance of pedestrian barriers. From the example, it seems pretty clear that the presence of freeways and super-arterials should also be considered as pedestrian barriers. Steep hills, undeveloped land, and large parcel properties (such as salvage lots, quarries, farms, airports, etc) should also be taken into account. All of this information is readily available in any city planning office, and could easily be taken into account by WalkScore.

  6. jon October 25, 2010 at 4:03 pm #

    they also need to have some way to be more picky with the grocery stores category to tell the difference between a true market and a convenience store or even a business with ‘market’ in the name… white house/black market and boston market are not markets. likewise restoration hardware is not a hardware store.
    additionally i think some categories (library, theater and non-everyday item shopping) dont need to be real close-by by foot so long as they are measured by ease of transit access. not sure how easy it would be to measure this though.

  7. Loosh October 25, 2010 at 4:15 pm #

    More flawed than WalkScore is their related offering TransitScore. When you type in an address, it only lists the transit offerings nearby, but nothing about those offerings. Therefore a rail line or a good inner city bus route that runs every 5 minutes looks the same as a bus route that only runs every hour 9-5 on weekdays. Whether there are transit offerings is very different from how good those offerings are – obviously – and good quality transit, not its basic existence, is what allows for life without a car. This software doesn’t take these variables into account, oddly, however, so it is lacking a very important aspect of its purpose.

  8. jon October 25, 2010 at 4:18 pm #

    i think maybe i’ll start a business, send me $30 and i’ll give you a full personalized analysis of your location, checking for true grocery stores/supermarkets, urban form and street design, transit service, topography, sidewalk presence, etc.
    my point is, this really needs to be done by a person who can give their subjective opinion based on real solid facts and data and a visual observation. formulas and algorithms dont cut it.

  9. Alex B. October 25, 2010 at 4:50 pm #

    @Leigh Holcombe:
    All of this information is readily available in any city planning office, and could easily be taken into account by WalkScore.
    Indeed, all of that information is likely located at any city planning office. But it most certainly is not easily taken into account. How will you adjust the weights for terrain, for example? Just how much should that matter?
    On the more technical side, integrating all of that data into an easily searchable and easily used format is exceedingly difficult.
    For all the criticisms of WalkScore – and they are certainly deserved, pointed critiques – too often the critics fail to address the single biggest positive aspect of WalkScore. That is an easily applied, rough cut at walkability. No matter the flaws of the dataset, in a big picture sense, Walkscore does (in my experience) give a pretty reasonable broad-based metric for walkability. It’s not a very precise metric, and we should always be mindful of the data inputs and the limitations of the outputs – but the big picture is nevertheless useful.

  10. Brent October 25, 2010 at 7:28 pm #

    The true benefit of Walkscore is in transforming how people view walkability.
    As the commenters above imply, walkability has two equally important components.
    One is the nature of the urban space — path connectivity, road cross-section (or even presence/absence of sidewalks), traffic volume and speed, protected crossing locations, and urban design.
    Of course, the flip side is that in order for a neighbourhood to be walkable, you need to have amenities within walking distance. Or, perhaps more critically, you need to be able to make the vast majority of your daily activities on foot — I argue that the most critical is a reasonably full-service grocery store, maybe in the area of 10 to 20,000 sq. ft.
    Historically, there has been a lot of focus on the former, and not enough on the latter. It’s as if designing a Portland-scale grid instead of suburban cul-de-sacs, or installing all sorts of street furniture, would on its own inspire walking trips. But it’s equally important, and maybe moreso, to make sure that there’s actually something to walk to.
    I suspect that part of the reason is that things like street pattern, right-of-way cross-section, are things that are easily measured and inventoried, whereas it’s historically been more difficult to record and measure amenities and proximity (especially in areas with high Walkscores), at least prior to the emergence of GIS data and Google Maps. Walkscore is the first tool I’ve seen to try and put a number to the presence and proximity of amenities, even if in a somewhat crude manner (in part due to a lower emphasis on the built environment).

  11. Alon Levy October 25, 2010 at 9:58 pm #

    My criticism of Walk Score comes from the diametric opposite end: it makes dense but amenity-deficient ghettos look more walkable than they actually are. Where I lived in Harlem, Walk Score transformed a corner bodega with a deli into a restaurant and another into a supermarket. I can’t imagine what it would do to a more walkability-starved ghetto like the South Bronx, where the only route to a park is circuitous and involves walking over Triboro Bridge.

  12. J.D. Hammond October 26, 2010 at 11:34 am #

    Alon: this, exactly. Not only do they define amenities down and confuse walkability with air radii, it also underrates the transit efficacy of highly linear neighborhoods like beaches, where everything is “on-the-way” by necessity. Ocean View in Norfolk, Va. might not be the most walkable place in the world, but it’s vastly preferable to parts of Virginia Beach with significantly higher densities of business but no real way for pedestrians to get to them.
    Still, it’s not an entirely unhelpful tool.

  13. Conservative Planner October 26, 2010 at 8:22 pm #

    Thanks for the insight and perspective. The hope of the post on Conservative Planner was to start this kind of conversation. It’s been frustrating to see planners attach themselves to the existing measure as something that is relevant when it has some very significant flaws if you look at Walk Score through the planning spectrum. As for heightening the awareness of walkability and its potential, it is an admirable effort and it’s good to see they are working to improve it.

  14. Jason Lally October 27, 2010 at 1:23 pm #

    I think there is an excellent opportunity here to catalog the nations pedestrian networks (sidewalks including widths) and barriers using the same or similar infrastructure as Open Street Map. We are working on developing the PlaceMatters Decision Lab which will bring together developers, practitioners, academics, designers, etc. who want to build the tools and techniques to make better decisions around planning and sustainability.
    Maybe this is one of many starter projects to get our pedestrian networks mapped in a national, reusable database. With the proliferation of iPhones and other smartphones, this may be within grasping distance.

  15. Jase October 30, 2010 at 10:38 pm #

    Getting data about pedestrian conditions out of planning offices is likely to be very costly. I think walkscore (and google maps for that matter) should crowdsource input on pedestrian cut throughs. Wikipedia has shown us that f people care about something, like a pet topic or a local area, they will make sure the online data is as best as it can be