Airline ratings articles are a dime a dozen, in large part because of the easy availability of government airline performance databases and regular customer satisfaction surveys spoon-fed by J.D. Power & Associates.
Not so for airports. Most airports are run by municipal authorities not geared for competition. Consequently, there are no airport performance compendiums a reporter can use to bang out a quick and comprehensive ratings. So when Consumer Reports set out on "Rating America's airports" (October, 2000), we knew we'd have to build the underlying database from the ground up.
Finding trends
Why rate airports, anyway? Preliminary reporting revealed that big airports dominated by major full-fare carriers were beginning to get their first competition from smaller secondary airfields, which had become metro market beachheads for low-fare Southwest Airlines. Nor was this phenomenon isolated; it was a trend taking shape in metropolitan areas from Boston to L.A., Miami to Chicago, New York to San Francisco.
Air travelers resigned to suffering long lines, long gate treks, and long delays at their same old airport may be surprised to learn that on some routes they can choose from a dozen different airports, if they take into account all their options at the departure, destination, and connecting cities. The choice, our ratings ultimately found, could make the difference between a tolerable, hellish or even a pleasant travel experience.
The benefits of building your own database are many: You get custom results that serve your specific goals. The drudgework of digging up all those numbers gives you greater depth of understanding and insight into your subject. Your original research is not dependent on available studies. Finally, you get exclusive data that your competitors will be hard-pressed to replicate.
Here are 10 tips for do-it-yourselfers based on our four-month airport project:
1. Define your standard of excellence. When Consumer Reports develops ratings, we aim to get to the heart of what consumers should be getting for their money. Airports are not a desired end per se, like a resort hotel; they're only a means to an end. But let's not be shy. Airports are an obstacle course between ground transport and what you're really there for: High-speed air travel.
Knowing this, we took our bold empirical stand: A good airport is one that gets travelers up and down from the sky as quickly, safely, and inexpensively as possible, with minimum hassle.
The benefits of building your own database are many: You get custom results that serve your specific goals.
2. Define your measures. How do you measure how well an airport meets this standard of quality? A Nexis search turned up few ideas or tools, and Web research of airport and airline sites revealed spotty, incomplete information.
So we used some common sense and our own good and bad experiences getting through airports for guidance. What determines how quickly you get on your way? Flight delays are one good measure. What constitutes hassle? How about a panicked search for an empty parking space as departure nears; rental car shuttles creeping through traffic-choked terminal roads; assignment to a Mad magazinesque Gate A1-Yea! Short walk! - that's really at the end of a three-mile-long concourse.
The airports
3. Define the universe. Which airports? Drawing on the Annual Worldwide Airport Traffic Report published by the Airports Council International-North America trade group (www.aci-na.org) we zeroed in on the largest U.S. airports in the nation's most populous metro areas. We also included smaller alternative airports with jet service in these regions if they were within 50 miles of a major aerodrome.
4. Organize! Because database builders have to deal with lots of individual bits of handpicked data, you've got to computerize those bits as soon as you get them. We plugged our data daily into an Excel 6.0 master spreadsheet, which allowed us to add, subtract, move, and modify data fields and records according to what the concurrent reporting for the article was telling us. The spreadsheet served three important organizing functions: 1) a behind-the-scenes storehouse of all information that might go into our final ratings; 2) an easy calculator for converting raw numbers into more meaningful comparative indexes; and 3) a project tracking tool that told which data were in and which were still missing.
Because database builders have to deal with lots of individual bits of handpicked data, you've got to computerize those bits as soon as you get them.
5. Use outside vendors. Price is a major motivator for switching airports, and we used it as a key ratings measure. We hired GRA Inc., a Philadelphia airline consulting firm, to extract real airfare data from U.S. Department of Transportation nine-track tapes and give it to us in a medium and format we could use easily: Excel files on CD-ROM.
The little-known DOT source database we tapped-Origin and Destination Databank 1b-offers an amazing wealth of information: A 10 percent sample of the 40 million airline tickets actually sold for travel within the U.S. each quarter. We asked GRA to compile an abundance of very specific airfare data, which we ultimately distilled to a "low fare" score indicating which airport in each metro area had the lowest average restricted coach fares to their most popular destinations. This rating was based on more than 17 million tickets sold during the 12 months ended September 30, 1999.
6. Get more data than you need. As with all reporting, you don't always know which information will wind up being meaningful. So it's always better to have data and not need them than to need them and not have them.
Not important
For example, we gathered what we thought would be important numbers from the Federal Aviation Administration showing each airport's rate of runway incursions-planes in unauthorized proximity to other aircraft, people, or ground equipment. Incursions were a hot topic at the time, having risen 71 percent since 1993.
But interviews with safety experts and examination of the data showed the risk of actual collision was vanishingly small. There were just 328 incursions out of 68 million takeoffs and landings in 1999. And no airport we examined stood out as having a significantly worse incursion rate than others. So our final ratings did not include this measure. No sense alarming readers with meaningless comparisons.
7. Use part of an existing database. We put together on-time flight arrival and departure performance data from a year's worth of monthly Air Travel Consumer Reports published by DOT and available online at www.dot.gov/airconsumer/index1.htm. The ATCR is ostensibly designed to rate airline performance, but some of the data series are also organized by airport, which we used to calculate the percentage of flights that arrived or departed 15 minutes or more late at each airport the previous year.
Site visits
8. Collect data in-person. To find out how far travelers must lug their carry-ons to the farthest gates, I traveled to 45 airports and walked more than 35 miles of concourses with a wheeled surveyor's measuring stick. Of course, this tour of U.S. airports was also used to get information on food prices and payphone charges, insight into terminal layout and design, digital video footage, anecdotes, and a firsthand understanding and feel for each airport.
9. Use a questionnaire. To count all the payphones, ATMs, restaurants, and other services, we developed a detailed questionnaire that we faxed or e-mailed to some 75 airport authorities.
10. Normalize and score. The raw numbers by themselves are not always meaningful. Sure, there are 40 restaurants to choose from, but that may not be a sufficient number if they're spread over seven terminals and you're prisoner at the end of one long concourse waiting to hear when your late flight will finally begin boarding. So instead of comparing the absolute number of locations where you can buy prepared food in each airport, we calculated the ratio of food outlets per terminal/concourse area. Similarly, we calculated the number of pay phones per 1,000 daily passengers, ATMs per terminal/concourse area, and available parking spaces per daily passenger whose travel itinerary originated at that airport.
Finally, Finance Editor Louis Richman, Statistics Director Michael Saccucci, and I worked together to translate all of the comparisons into the familiar Consumer Reports ratings symbols (known around our Yonkers, N.Y. headquarters as "blobs") to help readers make fast and easy comparisons.
Jeff Blyskal can be reached by email at blysje@consumer.org