A New York Times article today highlights the perennial misunderstanding embedded in how transit agencies typically measure on-time performance.
By official accounts, 2009 was a banner year for the commuter railroads that serve New York City. Of all the trains that ran last year, the railroads said, nearly 96 percent were on time — one of the best performances since they began keeping records.
But the reality, as nearly any rider would tell you, can be considerably different, and vastly more frustrating.
On weekday mornings, 1 in 10 trains entering Pennsylvania Station arrived late, two-thirds by 10 minutes or more. At the peak of the rush, from 8:30 to 9:30 a.m., about 25 percent of New Jersey Transit trains entering Manhattan arrived late; about 2 in 5 of the late trains were tardy by at least 15 minutes.
The explanation, of course, is that peak hour commuter trains are most likely to be late, because they run when the system has the least margin for error. And since most riders of commuter rail are on the peak, most riders experience the system at its least reliable.
In short, 96% on-time performance doesn’t mean that 96% of customers get where they’re going on time — only that 96% of trains did, counting nearly empty trains at 10:00 PM when there’s plenty of spare capacity on the rails, and a lot fewer causes of delay.
It would not be hard, of course, to calcuate and publicize the actual on-time performance for customers. Just multiply each train’s performance by its ridership. The result, of course, would be a much lower number, but one that would give you a better indication of the odds that you’ll be delayed when you use a service.
This is why other places (UK, I think) use metrics like passenger delay minutes. Or you can even just look at peak OTP. And keep in mind that “on time” means “within 5 minutes and 59 seconds of scheduled arrival time at the last stop”, which works okay for inbound peak commuters but pretty badly otherwise. And really not so well for peak inbound commuters either, since there’s always a couple minutes of schedule padding before the last stop, so the train running perfectly would mean it’s 2 minutes early.
This subject does point to one annoying asymmetry between roads and transit.
When someone’s late for work because the freeway is jammed, it is frequently blamed on insufficient capacity.
However, when someone’s late for work because the bus or train is delayed–it’s because the transit authority is incompetent.
What’s also interesting to note is that those measurements are easily deceiving. Till recently the Dutch Railways measured punctuality where a delayed train is one that is more than 3 minutes late, while most of Europe’s railroads measure with trains not being late till they’re 5 minutes late.
The “real on-time performance” is not really real either: if you have to switch trains/services, if your train is delayed, you may miss a connection and still be late. Now in subways and high frequency transit that’s not as big an issue as when your frequency is slower…
Coming from Beaverton to downtown Portland to hear Jarrett’s talk, I kept running into the opposite problem–the dang MAX train kept pausing at time points, so not to get too far ahead of schedule. 🙂
Then there’s also the issue with the definition of “on time”. For instance for King County Metro the definition is within 10 minutes of the scheduled arrival time at the route’s terminus. On a long commuter run that may make sense, but on a local routes with the full trip taking 30 minutes that trip could take 33% longer and still be “on time”. I’d prefer a system that takes into account arrivals at time points and defines on time with regards to the total trip length.
The solution is transparency. If the transit agency were to give out the raw data (actual arrival times of trains) then someone could make a website with graphs and such which let you customize what on time means to you.
I dont know of any transit agency that does this.
The MBTA does publish a monthly scorecard that breaks down delays by commuter rail line, which is a start, but could be better.
While all the data isn’t made public, Melbourne’s performance regime includes the concept of “passenger weighted minutes”, in an attempt to distinguish the difference between a 10 minute delay a train carrying 1000 people vs a train carrying 200 people.
More than 4m:59s behind schedule is considered “late” for metropolitan trains, 5:59 for interurban, and 10:59 for long distance.
J: You can get actual arrival times for some systems. In the San Francisco Bay Area, for instance, Muni has a public API to give you the last-known location of each of its vehicles. BART is a little less direct but will tell you, for each station, what trains are currently loading on the platforms and how many minutes away other trains are.
J and Eric. Anyone can monitor reliability now. See here: https://www.humantransit.org/2010/06/now-anyone-can-monitor-reliability.html
My favorite is how my local transit counts buses as being late when they arrive more than 5 minutes behind schedule… but with only 3 minutes to spare at a lineup, to passengers, our performance is much worse than 86%
Yeah, the MBTA now has a realtime feed of the location of all buses, via NextBus’s servers (though fetching ‘all buses’ instead of ‘buses on route N’ isn’t supported, it does work). I’ve got some software grabbing it frequently and archiving it for some analysis when I get a supply of round tuits….
Jarrett, multiplying train performance by ridership is a step in the right direction but not enough: you have to model missed transfers too.
If a train is late but the passenger still catches the connection, it might be 0 passenger minutes delay – whereas if a 30 minute headway connection is missed because of only 8 minute delay then this is in fact a 37 minute passenger delay!
It is not that easy, after all 🙂
Btw “on time” means a different positive number for different countries. 3 minutes in Copenhagen, 5 minutes in the Netherlands, 6 minutes in Belgium and Sweden, 2-3-5 minutes in Switzerland. Without considering this, it is hard to compare.