Just how prepared are major U.S. cities to face a terrorist attack? Can we assume places like New York and Washington, D.C are more up to speed than, say, Milwaukee or Columbus?
The questions were being posed by editors at CNN.com as another possible follow up to events of Sept. 11. Like several news organizations, we had completed a computer-assisted reporting project that focused on the airline industry and security at the country's busiest airports, and there was a desire to similarly rank, or at least gauge, the performance of major cities.
Terrorism experts, who by all appearances had multiplied in number after Sept. 11, were insisting on two things: no city is immune from risk and no one is fully prepared to handle the fallout of an attack. So folks couldn't help but worry about their own safety as well as suddenly wondering if their city was schooled to deal with everything from biological to chemical threats.
To assess the preparedness of the largest cities, CNN.com undertook a computer-assisted reporting project last fall that produced "How Prepared Is Your City?" on the Web site, a 30-minute special report on "Wolf Blitzer Reports" and also a TV package by CNN Homeland Security correspondent Jeanne Meserve.
The early challenges were plenty. First, we were determined to produce a thoughtful package that would both enlighten and address people's concerns, yet at the same time wouldn't serve as a blueprint for would-be terrorists and criminals. We found this also to be the top concern of the experts who were interviewed or assisted us in evaluating the cities.
This ongoing dialogue with a team of experts proved invaluable, as they often had first-hand knowledge of the preparedness efforts and vulnerabilities within a number of major cities. We also were able to seek their input on the statistical data, federal grant money and such criteria that would best help in evaluating the work done by city officials leading up to Sept. 11.
Narrowing the cities
OK, but how many cities could we review in depth? We thought initially 50, until deciding it would be too overwhelming and time consuming a task. And when we considered 25, we struggled to justify eliminating important cities like Miami and Atlanta that barely missed making the cut.
Ultimately, we reviewed national crime statistics reports and developed our top 30 cities by the criteria that cities had to have a population of at least 350,000 and also employ at least 1,100 full-time police officers. Even then, to include as many cities as possible, Dallas/Fort Worth and San Francisco/San Jose were combined rather than evaluated individually. We made sure to present separate data on each city in our final report and, as it turned out, our final rankings for these cities would have been the same no matter which way we did it.
That bit of information would prove helpful in addressing challenges from Fort Worth and San Jose officials, who argued that their respective cities had their own individual issues and didn't care to be evaluated with their neighbors.
Once we were comfortable with our list of cities, we dug for every shred of relevant data and public record that was available. The analysis would focus on four crucial categories: transportation (access to and from emergency sites); hospital/medical; federal preparedness (an evaluation of funding and training programs); and emergency management.
True preparedness is often deemed best achieved where the local, state and federal governments - as well as members of the private sector - work as partners. But we recognized a challenge in preparing for emergencies was also getting access to available resources.
The Centers for Disease Control and Prevention, we found, awarded $120 million to states during the past two years to upgrade public health's capacity to respond to bioterrorism. But only four cities received separate grants, including New York ($1.14 million), Chicago ($902,089), Los Angeles ($850,275) and Washington, D.C. ($570,317).
Finding, analyzing and weighting the value of consistent, statistical data that could be applied, in our four categories, across the 30 selected cities, would be a key to making this study work.
Surveying experts
Early in the project, however, it became apparent that statistics alone were not going to tell us the full story of a city's preparedness. Some of the information that would be valuable in evaluating cities, such as details about pharmaceutical stockpiles, is appropriately guarded.
The experts stressed the need to weigh intangibles, such as how the uniformed services work together as well as the level of commitment from the private sector to participate in training. So while accumulating the data in each of our four categories, CNN.com also provided local emergency management directors with a 10-question survey to evaluate staffing, training and such key questions as:
Who does your city's emergency management director report to? A. Mayor's office/city or county manager B. Police department C. Fire department D. Other
The number of your full-time staff dedicated solely to emergency management? A. More than 10 B. 5-to-10 C. 1-to-5 D. 0
In the current fiscal year, the amount received by your department for terrorism planning, training and equipment? A. More than $1 million B. $500,000-to-$1 million C. $100,000-to-$500,000 D. Less than $100,000
All of the data accumulated by CNN.com was later shared with the local emergency management directors, who were offered the opportunity to comment and report any corrections. In most cases, it required multiple follow-up calls and faxes but it was crucial that officials confirm the accuracy of data that would be used in the city evaluations.
Eventually, we would provide our empirical data to six law enforcement, security/terrorism and emergency management experts asking them to rate the cities in each of the four categories, using a one-to-five scale (one being best prepared). The Disaster and Terrorism Assessment Survey given the experts was a 22-page document that included key potential vulnerabilities of each city, general city information such as the number of police officers per 10,000 residents and then specific data compiled within our four categories.
Such a scale is recognized by, among others, the National Institute for Computer-Assisted Reporting, as a common grading system within academia, the media and law enforcement. The U.S. Department of Justice provides the same scale for a self-assessment by states applying for domestic preparedness grants. The most comprehensive was hospital/medical, which included 18 items ranging from physicians and emergency outpatients per 10,000 residents to whether the state employed a public health veterinarian and the number CDC-sponsored "disease detectives."
The final overall assessment, using Excel spreadsheet software, of the 30 cities was based on the simple average of the experts ranking, with CNN.com weighting the hospital/medical category 1.5 times because of its importance and the volume of relevant data. Cities were judged as being most prepared, well prepared, prepared or less prepared.
While reviewing the data, we stressed the importance for the experts to rely on their own experience with the cities. So factored into the evaluation was their awareness that some cities face a greater risk because of such things as location, commerce, landmarks and hosting major events.
In the end, only New York, having earlier experienced attacks on the World Trade Center, was judged most prepared. We did a separate story looking at New York's performance and the lessons learned.
The major knock against New York came in the transportation category, where experts cited the potential for gridlock - a factor in getting responders to the scene as well as transporting the injured and ill to medical facilities.
Several other large cities also fared poorly under transportation - notably Boston, Philadelphia and Detroit. The panel judged them overall less prepared, joined by Las Vegas, Milwaukee and New Orleans. CNN.com shared the rankings with city officials before publishing and produced a separate story detailing heightened preparedness as well as steps taken by the individual cities since Sept. 11. Our analysis also found:
While cities are better prepared than prior to Sept. 11, significant improvement is still needed in public health, emergency response, evacuation planning and inter-agency communication.
Las Vegas and New Orleans, two major tourist/convention cities, are located in states that do not employ public health veterinarians, who could be helpful in the early detection of a biological crisis. Nor did the cities have a CDC-sponsored disease detective.
Cities faring best are those like New York who routinely host major events or traditionally face natural disasters, such as Florida and California cities.
The two-month project produced eight stories for the Web site. Web site visitors could rank the cities on different variables and see how their cities compared to others.
Mike Fish can be contacted by e-mail at mike.fish@turner.com