The Information Gap

Second in a series of essays humbly titled “Fixing Journalism”

An email list for news researchers that I subscribe to recently featured a discussion on the kinds of skills necessary to get a job in the news library, which performs research for reporters and editors. In several of the posts, managers said that they were looking for people with CAR skills in addition to the traditional abilities of searching records and providing important background and missing information.

This is, to my mind, an acknowledgment of the growing challenge facing newsrooms: information management. Whether we’re talking about timelines, interview notes or databases, newsrooms have ever-more information in digital form and we’re doing too little to index it, manage it or make it broadly available.

This situation is exacerbated by what I see as an industry trend in which the people who write the stories are removed little by little from the information and subjects they write about. More frequent deadlines, increased competition and the ability of the Internet to spread information quickly demand that reporters be able to assemble a broad picture quickly, many times with the assistance of researchers.

The way that we gather information has changed dramatically over the past decade, and reporters and editors have not kept up with this revolution. The wealth of material available through online databases in particular and also on the Internet means that reporters have access to much more information than they did in years past.

So, for example, if a reporter is seeking to write a profile of a relatively well-known person, he or she could request a “clip-job” of what other papers have written to see what else has been published. They can request searches of public databases and other records. In most cases, this is a good thing; more information is hardly a negative. But with increased access to information comes a greater need for better management of it.

The real trouble comes when research tasks are repeated within newsrooms and the cumulative value of that work is either locked away in individual computers or simply thrown away. In the worst of cases, we end up doing the same task more than once and while the information gets to its consumer, nobody in the newsroom learns on a permanent basis.

Institutional knowledge in newsrooms now largely resides inside the heads of reporters and editors or within some files that few people can access. Aside from the incredible inefficiency, this model also suffers from the “hit-by-a-bus” scenario – what happens when key people leave suddenly? The case for sharing information broadly inside newsrooms has never been better than now, when we have access to so much data.

Newsrooms collectively face a couple of situations from this massive availability of information. The first I’ll call the Google problem. This isn’t Google’s fault, but rather an unintended consequence of its influence and ease of use. The Google problem can appear in several ways, but one of the most common arises when reporters and editors attempt to use the newsroom as their own search engine, for example by asking for sources or seeking out information through emails to the entire staff.

This is also inefficient, particularly when the newsroom in question is a poor approximation of a Web search engine: results may vary wildly and can be easily skewed by the respondents to such queries. If reporters want to use the newsroom like they do Google, we need to have a better system of harnessing the knowledge within it and making it available to reporters in a more systematic way.

Reporters also come to love Google’s ease of use and think that all archives or databases should be as simple to use. But most newsroom resources have their own search syntax or begin with a complex interface or suffer from neglect. People turn to Google because it’s simple and it doesn’t change much. We need to make our information much easier to search.

The second problem is that many reporters don’t keep pace with what they need to know about searching databases. Newspapers and other media organizations need to do a better job training reporters and making such instruction an important and recurring part of the job. This is crucial because it is a cultural problem rather than a practical one. It isn’t that reporters can’t do this, because many are expert at finding hidden gems within paper documents like transcripts, budgets and court records. Some of them either opt to depend on other people when available or don’t keep up with search skills, and so the layer separating them from raw information only grows. Any such layer increases the risk of misinterpreting or just flat-out missing good information. In an age where a growing set of readers have both experience in finding information online and access to some of the same tools, newspapers need to get this right to maintain and enhance their credibility.

Finally, there’s a very important reason to adopt better information management practices: we’ve got to keep pace with the people and organizations that we cover. We see some of the most dramatic examples of the information gap in political coverage, where the consultants and party officials that papers cover have used technology much better than we have. We see it in baseball, where executives rely on data analysis as newspapers continue to print boxscores and offer only occasional insights based on empirical evidence.

So how to close this gap? Let’s start in the news library, where a lot of information already is generated or compiled and where the culture of collaboration is typically strong. Create a network for researchers to post or upload both their information-gathering strategies and the best results, and allow others to view them, comment on them and share them within the newsroom. It’ll save duplicative efforts and teach the library staff best practices that can help spread the word throughout the newsroom.

Then, when reporters ask for information, researchers can check to see if it already exists (or needs to be updated) and find out the best ways to do so. They also could begin to standardize their delivery methods so that the information they pass out is archived and searchable, if not categorized in some way. It need not be a universal archive, but perhaps within a single section, team or beat. If reporters want to print out the results, fine. But having a permanent archive will benefit the entire paper, not just a single reporter.

An additional benefit is that the record of information requests and results could be available to the reporters and editors, so they can see what was done and get more involved in evaluating search strategies and sifting through results. But we need to be clear on using a computer for these tasks: money that the newspaper spends on costly databases and people to search them should benefit the entire newsroom, and the best way of ensuring that is to keep an electronic record of that process.

The platform to accomplish this already exists: the Web. It’s greatest strengths are simplicity – reporters know how to use a Web browser – and flexibility. We can make data appear in different ways, allow various types of searches and, perhaps most important, reporters can be given a share of ownership in the information. That, more than anything, will motivate them to use a system of collaboration. The Web also is accessible from the office, home or the road. It doesn’t require purchasing and installing expensive software, and it cuts down on specialized training.

But the best reason for using the Web to distribute information internally is adaptability. Few software makers design products specifically for newsrooms, which can have unique needs. We should steal the best ideas from the growing suite of applications that address information management. Consider the manifesto of Backpack, a new Web-application from a group of designers and information architects in Chicago called 37Signals. It’s a dream come true for the often sloppy world of journalism:

Your information is everywhere. Scattered across notebooks, emails, post-it notes, and god knows where else. With Backpack, there’s a better way. Now Information is easy to organize, centralize, and retrieve. Information works for you instead of vice versa. Backpack adjusts to organize things your way. It’s a blank slate that offers you less structure and more space. Easily fill your pages with anything you want, from notes to to-do lists to images to files. And then group them however you want, wherever you want. It’s organization the way you want it.

Customized database applications like Lotus Notes or FileMaker Pro can work in certain circumstances, but nothing works like the Web. Imagine if, instead of newsroom Intranets, we had internal Wikipedias in which reporters, editors and researchers worked together to preserve and improve the paper’s institutional knowledge? Some of the time now spent by researchers answering requests could be handled by a living Intranet updated by the people who know best would enable them to spend time on more complex (or urgent) tasks.

The basic tasks of journalism are not threatened by the Web or its principles. Indeed, they provide newspapers with an additional boost, if reporters and editors grasp the long-term benefits. But the goal of journalism – accurate, informed information – can be done easier when reporters, researchers and editors are working closer together, all sharing in the process of finding and evaluating information.

We will always have anonymous sources and reporters who hold their secrets tightly. Newspapers should not seek the perfect internal archive but instead try to build the best they can. Some areas may need to be password-protected within teams or sections, with a clear point-person who can grant acccess or provide details. But these are exceptions and should not be the rule of thumb throughout the newspaper. Instead, we should share information with each other and with the public. It’s time for newspapers to stop treating the information age as simply a topic of coverage and instead make it a central part of the newsgathering process.

At a recent computer conference in California, Adam Bosworth of Google told the audience that he wished to move beyond the current state of Internet searching. Search engines like Google return information, but he wanted a Web search engine that would provide content – things with specific meaning and value. That’s what people want from newspapers – meaningful content, not just information.

So how to get better content? Well, you could make sure that newspapers hire only the best and brightest and then give them the freedom to report widely and consult with the best experts in their fields. Again, we’ve got a scaling problem – in this scenario we’ve got a handful of journalistic stars working for an even smaller number of outlets, and underpaid and overworked folks down the ladder. This is not a long-term solution – journalism at the lower levels presents an inviting target for niche businesses that can capitalize on the strengths of the Web.

We must, then, be better than we currently are at every level. Good information management practices benefit the reporter through better recognition of trends and relationships. They benefit newsrooms in general through increased efficiency and wider access to common data. And they benefit readers by giving them better-informed journalism.