Three Modest Proposals for Online Journalism's Future

If you follow me on Twitter, you probably noticed that I spent my lunch hour at the Open Society Institute today for a talk on "The Future of News" by Paul Steiger, the longtime managing editor of the Wall Street Journal, who is the head of ProPublica, an "independent, non-profit newsroom that produces investigative journalism in the public interest." It was a mostly gloomy session, framed by the news that 11,000 professional journalists have lost their jobs in the last two years, and all the bad news currently coming out of the newspaper industry.

You also probably know, if you follow me on Twitter, that I was gnashing my teeth for much of the time, frustrated by several questions from audience members who wanted to focus on shoring up the failed business model for today's dying newspapers (mostly by charging for content) instead of figuring out what comes next and how we insure that intelligent, inquisitive and informed reporting and analysis continues to enrich our society. Almost no one in the room seemed cognizant of the arguments being made by thinkers like Clay Shirky and Steven Johnson, either about the inevitable death of print newspapers or the vibrant rise of a rich online ecology of topical, and often deeply investigative, journalism.

In fairness to Steiger, I should note that he personally seems well aware that, as he said at one point, the horses are already out and attempts to slam the barn door shut would probably not work. He also spoke optimistically about emerging forms of online media, and more than once expressed hope that crowdsourcing, as practiced by OfftheBus (whose chief wrangler, Amanda Michel, he has just hired at ProPublica), could be a viable model for pro-am collaboration.

I left the lunch a bit depressed, but as I thought things over on my subway ride back to the PdF office, a few ideas began to coalesce in my head about how we could perhaps foster a stronger ecology of investigative journalism in the new online environment. So, instead of beating a dead horse (to mangle my metaphors), here are three ideas for projects that could help sustain investigative journalism however it is practiced going forward:

1. In an Age of Mass Participation, Make News Easier to Make Together
We need better tools for mass collaboration. The OfftheBus model, as described by Amanda in a terrific essay she wrote for the current issue of Columbia Journalism Review, is highly labor intensive. About 12,000 people volunteered time and shoe-leather to the project. As I understand it from her article and from talking to her, there were all kinds of headaches involved in managing this outpouring of volunteer energy. She writes, "the challenge was not persuading them to sign up. It was figuring out what they were willing and able to do after that, and then cost-effectively coordinating their efforts so that they added up to real journalism."

The future will be reported by more of such dynamic pro-am collaborations, but few of the existing tools for working on a project together (such as parsing a long document and sharing notes) work very well for small groups, let along large groups. To take another, perhaps more esoteric example, on OpenCongress.org, over the last year, more than 40,000 people have commented on legislation to extend unemployment benefits. They're congregating there by accident. People go online searching for information about extending unemployment benefits and discover that OpenCongress's page for that bill allows comments (unlike the official Library of Congress page). First they shared information about what was holding up the bill, but now if you look at the thread the comments have evolved into a self-help group of people helping each other figure out how to get their state's unemployment office to release their benefits as quickly as possible. If this is what a random, unorganized group of people can do on a threaded discussion, imagine if we had better tools for large groups to sort themselves into teams, or to track each other's work.

2. In an Age of Data, We Need Easier to Use Visualization Tools
Earlier today, my colleague Nancy Scola tweeted at the start of her SXSW talk on White House 2.0 and open government that "Data is the new 'plastics'." I think she's right. Whether it's all the government data being liberated by my colleagues at the Sunlight Foundation, or being shared by government itself, or the new APIs that a handful of forward-thinking news organizations like NPR and the New York Times are making available.

But how to make sense of all this data? I think we need easier-to-use tools, and that some targeted investment in fostering better tools might produce a huge payoff. Blogging took off because free tools like Blogger made it so you didn't have to know how to write .html code to publish a web page. Videosharing took off because YouTube made posting video online as simple as attaching a file to an email. But data visualization is still too hard. I want to be able to grab a database and toss it onto a Google map without having to know how to convert it to .kml. I want to be able to play with MotionChart without needing a design wizard to decipher Google's missing interface for it. The only reason I was able to make this chart below showing the growth over time of the financial sector in financing campaigns was because I had the help of Kerry Mitchell, one of Sunlight's crackerjack designers.

When I brought up this complaint at Transparency Camp, some folks suggested trying out GeoCommons, and I was pleased to discover that that site offers some relatively easy tools for quickly mapping data. But I think we are just barely scratching the surface of what is needed if we want to see an explosion of dynamic, data-driven (3-D) journalism online.

3. In an Age of Mass Information, Make it Easier to Find and Share Investigative Reporting
Steven Johnson's hyperlocal media site Outside.in currently tracks news, views, and conversations in 11,860 towns and neighborhoods. If you search on the words "political scandal" you get more than 3,000 results. I have no idea how many of these stories are of national significance, or if perhaps there are hidden connections between seemingly disparate local events. Nor could I tell you how many of these stories are being generated by old fashioned media outlets, or if they represent the work of new media entities like bloggers. Similarly, if you search YouTube for recent videos about "political scandal" the top current result is an audiotape of a top aide to the mayor of Mt. Vernon, NY discussing all kinds of local corruption.

Does anyone have any idea how many stories, blog posts and videos are published every day that shed light on local, state or national corruption? What about corporate crime? Last week, Cheryl Phillips, the president of the 4000-member Investigative Reporters and Editors organization, mentioned to me that her paper, the Seattle Times, had done a big expose on the improper sealing of court records, an endemic tactic often used by businesses to hide embarrassing information from the public, and often occurring without justification. She has a feeling there's a national trend worth uncovering, and perhaps certain multinational corporations are using this as a tactic across state jurisdictions. But she has no way of connecting those dots.

Well, maybe some smart information hackers can pull together some algorithms to scour the web and help us see the hidden patterns to what old and new media outlets together are publishing online. For example, how about a "Memeorandum" or "TechMeme" that aggregated attention around content containing keywords like "scandal," "corruption," "political", "corporate", "Congress" and/or "statehouse." Government transparency is going to spur far more coverage of government and/or contractor misbehavior. How will we sort and prioritize this information? If we want more investigative coverage, we need to develop better filters to help us zero in on the good stuff and also find hidden patterns in seemingly disconnected stories. Plus, better filters might eventually channel more traffic toward producers of good investigative journalism, and thus prop up whatever business model they may be employing.

There are undoubtedly other answers to the question of supporting serious reporting online (focusing on things that people care about, as is done by the blog Consumerist, might make more sense than trying to churn on eye-glazing six-part exposes that win old-fashioned journalism prizes but few readers, for example.) But I guess what I'm roughing out here is an agenda for technologists who want to turn their talents to strengthening democratic culture and practice. Can you make a better collaboration tool, or an easier-to-use data viz tool? Do you have one that we don't know about? Do you think it makes sense to build a ScandalMeme or an Investigandum? I'm talking with my colleagues at Sunlight about all of these ideas, and hopefully we can make some contributions in these arenas. But nothing beats learning from people smarter than you--so if you've got something to add to this discussion, please chime in or spread these questions (and answers) where you think best.

Comments

We're only at the beginning

Micah, I think we're just at the beginning. Back in 1994, the only people with websites were those who figured out HTML. Today, there are an overwhelming number of ways to get a site going: a blog on wordpress.com or blogger.com, a wiki, etc.

Also, today, the only people with solid maps are those who speak the tech talk. But give it a few years. And as we saw at Transparency Camp, there are many experiments underway now, each with a different approach to mass collaboration.

I'm very interested to see where this goes, not just for journalism, but for governing. That'll be a critical set of tools for really using the power of the net to choose the best regulatory path, for example. We need to break away from the old paper model of person A (in gov't) writing a proposal and people B-Z commenting, then A going through them.

I'm not convinced a wiki is the way to go; too many disparate viewpoints to really write collaboratively. But maybe we could use a suite of tools? One set to identify the various viewpoints, then a wiki or whatever to help each group coalesce their thoughts.

I see lists

Reading this post, it strikes me that this may be a good set of topics for some wiki-style tracking of what you're describing.

We're setting up some relevant tracking on the new OpenCongress wiki:

http://wiki.opencongress.org/wiki/Project:Transparency_Hub

Some things that may be worth building persistent pages for:

- visualization tools
- distributed journalism projects
- mass collaboration tools

Pay Per Views

Micah,

Here's what's really seeming to bother you, if I may be so bold: Paul Steiger, formerly of the more conservative Wall Street Journal, and his more mainstream all-star cast of board members and journalists, are not in the leftwinglittlemagazine-o-sphere or the lefty blogophere you want and need and fervently wish to support for reasons of ideological avocation. Maybe there aren't enough buyers for the leftwingosphere, I don't know. What happens when the cash runs out in either foundation model, right or left? It might be that Steiger's model, which only costs $10 million a year which isn't THAT much, can be done for half or a tenth as much by smaller and leaner and less famous operations -- good! In fact, many newspapers still live. In fact, they already went online. So it's not over til it's over.

There's nothing wrong with a business model if it works. The thing about business is that when it doesn't work, it folds and finds new models. Which it is doing. Moving online isn't death. Despite what Clay Shirky keeps ranting, micropayments haven't been tried, nor have subscriptions. The Times could start charging, for example, to comment, and find that more people will pay to publish their own comments than they'll pay to see other's content, following the concept of MMORPG and online community premium subscriptions.

Crowdsourcing? Please show me a crowdsourced or wikicultured source on foreign news that isn't about, oh, Code Pink and the Gaza strip or something that will have both a predictable obsession and a predictable bias.

1. We don't need better tools for "mass" collaboration. Good God, Micah, give up all this Marxist and Soviet fascination with "the masses". There are no masses, just because a million people can watch a YouTube. How about better tools just for collaboration with people who are like-minded, but with the possiblity for democratic correctives from those who are not like-minded -- which, at the end of the day, is what civic culture is all about?

That isn't the average Drupal or geekrun website, in which the tools exist mainly to blog out and mute people or make them jump through hoops to register and have their IP address harvested. And we don't need 40,000 comments on a bill that nobody can read through. We need someone to watch the 40,000 and report on the trends, and that person will have to be educated and connected and not muting outside agitators. We need I.F. Stone's Washington Weekly -- the Izzy Stone who will read the bills, even read the 40,000 comments, but then say something coherent in his weekly newsletter.

A few examples where you might find good Samaritans aren't scalable in the YouTube geek insolence culture and the geek-control culture that responds to it; the tools have to get easier for groups of like-minded to work easily by not be able to endlessly control *commentary* (note that your big friend Clay Shirky does not allow comments on his blog whatsoever, only his culled trackbacks. That's one way to solve the problem, Comrade!).

2. Visualization is fun, but stupid. Tag clouds do not add knowledge. They make for silly unintended interpretations. Charts and graphs are not the same as analysis of news. In fact the tools of Microsoft Office Suite and Google docs are now such that a 14 year old can easily make pie charts and tables and graphs. You're just not 14.

3. It's not that we need to "find and share" investigative reporting only. There's a scarcity of the right kind of unbiased reporting, and too much blogging on the left masking as "investigation". (The right seldom does investigations in the same enthusiastic way).

There is a real disconnect between the supply (leftwing blogging and Tweeting) and the demand (mainstream and conservative readers schooled in conservative or mainstream media which you cannot convert by persuasion).

What needs to be done is create better interfaces for expressing the social demand for investigative reporting and the social rating and voting on the results of such reporting, not in the "vote yes" sort of way one often finds, or the Digg gamed way of jacking up or down stories you don't like, but the newscred.com way of being able to rate and comment both on sources and individual stories -- and ultimately, have the commentators be accessible to sharing in an RSS feed that could appear like the new shared alltop.com so that a more valued set of commentators and raters emerge than you can get from a Digg.

Let's say someone really wanted to connect the dots on improper sealing of court records, and not merely cite it in a facile way as some purported civic good that "somebody" should do. If there were a craigslist of want-ads for news divided into city and categories, then those who really do care about sealed records can ask for the skilled reporter to cover the story. Those skilled reporters looking for work can put up the ad offering to research it. Could a kivo.org micropayments system paying a reporter in $25 and $5 bits work? Probably not, but no one thing will work, and lots of things can be tried without prejudice -- the $10 million Paul Steiger experiment and the $10 microreporting experiment can all be tried once you stop demanding that news be free or that news stop being a business that can't have a capitalist market check and balance on it, and has to be subsidized as propaganda by the government or by liberal foundations.

4. Your notion of using the intention economy -- the searches of what people are interested in which reflect their intentions to some degree -- as a way of ordering news strikes me as pretty crappy. People look up all kinds of things. You find this out when you see all the Google searches that lead people to your blog for all kinds of reasons. If what you suggest were viable, Google would long ago have created a news front page as a landing spot that had intention searches guiding the news. But they don't. Because they don't want to look like Yahoo Answers, which has top stories about what to do when your cat barfs.

5. Technologists can't spur democatic culture and practice by welding their leftwing views into the tools to skew the news and get predictable results. That is, they can for a time and get a burst of coverage in old media doing that, but it wears off. People won't pay for it. Technology isn't science anymore; one of the things wrong with technology know is that increasingly in fact its divorced or separated from science. Science, which doesn't ignore feedback by muting it, will tell you that most people are anonymous and insolent on the Internet in a setting like YouTube because they still do not feel empowered by new media, but feel intimidated by it because it is biased and still elitist.

Expertise and money

These are all good ideas, Micah, but they still don't solve the problem of the disappearance of the loss of the beat reporter. The production of useful knowledge in the face of vested interests that want to keep knowledge to themselves requires independent investigators who become intimately familiar with their fields because of day-in, day-out involvement. And it requires the kind of analysis of that information that can come only from independent observers who know the whole history of developments and debates and scandals and personalities and interests. Developing that kind of expertise requires someone to fund reporters who spend a lot of time just hanging out, chatting people up, and reading whatever they can get their hands on. Not that the daily press has always done this well, especially in recent years, and it's true that some of this can be taken over by researchers in academe. But ultimately someone--whether its people out to make a buck, non-profits, or the state itself, is going to have to provide the funds to pay people to be beat reporters, or whatever the new media equivalent will be called.

http://southjerusalem.com

Some great ideas Micah

I've been writing about this a bit at OpEdNews.com and huffingtonpost exploring the state of journalism. We, meaning democracy, need journalists but not necessarily newspapers. We need investigators and analysts, muckrakers and data diggers, catchers, scoopers, finders, see-ers and detectors. We need the work of journalism. That needs to be funded somehow. Printing newspapers is fading as a viable economic model. But the WAY to get the news out to people, the technology to take content and enable it to be found by the people who need it is better than ever. If anything, the use of the newspaper, pay to read model is hampering the process of enabling news and the product of journalists to be found.

There are blogs and wikis, on-line media sites like propublica and OpEdNews.com and huffingtonpost's off the bus and newsvine and other sites offer ways for independent writers to get their information and idea out. OpEdNews.com already has a group support function that enables groups to join and share information, and we also have pages for every nation in the world, every state, county and major metro area in the US and all provinces in Canada and Australia.

We had dozens of articles by Nepalese freedom fighters before democracy was achieved and the monarch stepped down. We've had numerous reports from The Pakistani regions where Bin Laden lives and routinely know about missile strikes, taliban actions and the like days before CNN reports it. Did it cost $10 million to build or maintain it? NO. We did it with member contributions under $60 K a year and a total of $21,000 in grants from sunlight, bread and roses foundations and a private donor. We also do polls, customized action pages, and we have some of the most sophisticated software available for volunteer editors to use to run an article submission queue and to deal with flagged comments.

What the USA and the world needs is a new way of thinking about funding journalism and journalists who can feed content to existing and emerging FREE content sites like the ones described above. I believe the model to use is the way the arts are funded. The government provides hundreds of millions in funding and should be spending more. Construction companies must include, in many locales, an art budget when building major edifices. Communities create arts councils, fairs and events to support and promote local art. We should do this with journalism. The government should invest a billion dollars to fund 20,000 journalists, for starters. Consider this an investment that will be paid back many times as the journalists uncover waste and corruption and educate the public. Communities should support local journalism efforts-- full and part time. Corporations should pay a small tax to support a fund that goes to pay for journalists that investigate those corporations-- to protect the investors from corruption, waste and abuses of power. Again this will be an investment that pays for itself many times over.

When I propose this idea, people have a knee-jerk reaction that it's a bad idea for the government to employ journalists. But artist who receive funding are not government employees. They are independents who get grants. The same can be done with funding journalists.

Here's where it gets interesting. Crowdsourcing and the wisdom of the crowds can be used to decide how much journalists are to be paid. Articles can be offered free through various feeds and access points for editors and users to post to their blogs to facebook, to digg, to all kinds of sites. When readers like the articles and value them, they can rate them and give the writers points that will accumulate. The writers with the most points, and this can be a basket of the many possible measures-- page views, email forwards, number of comments, ratings on different measures, favoriting, number of fans... At OpEdNews.com we use most of these as a way to enable our readers to navigate using different forms of "popularity.

The bottom line is that funding investigative journalism is just about sure thing-- that it will result in savings and loss prevention-- monetarily and societally-- far beyond the money spent. We need to jump on this soon. The old system is already failing, on its death bed and no longer doing the job we need it to do and which could be done so much better.

Just because all the possibilities have not yet been sorted out yet doesn't mean we should wait until the last final death throes are over before doing something to replace our archaic media system. We need to start talking about and funding the new journalism NOW-- and that may mean we have to walk the gauntlet of accusations of socialism that Universal healthcare is also subjected to. Just remember that the people casting the accusations are the ones who benefit most from the lack of light caused by the death of the old media. They have to be taken on and overcome. They will decry and attempt to diminish the light-- it's truth, it's purity, it's reliability-- because they do their business better in darkness. They are losing. They will fail. But we must take action and not assume that as the newspaper industry dies, journalism will simply, naturally adapt. We have to make the new world of journalism happen.

At OpEdNews.com our readers and writers-- 27,000 registered members, 400-800K unique visitors a month-- describe us as "the People's media." The future is bottom-up journalism on the web, accessible by all forms of media technology.

Rob Kall
Founder, publisher, OpEdNews.com
Host, Rob Kall Bottom Up Radio Show, WNJC 1360 Philly Metro Area
Regular Contributor, Huffingtonpost.com

Visualization tools

ManyEyes.com is pretty cool for do it yourself data visualizations. As more people use it, more large data sets will become available to play with for those who don't have data to upload themselves. Great post!

E-Votes for Europeans

Votes for Europeans are possible: "http://www.we-change-europe.eu"

"We Change Europe" wants to offer the opportunity of getting involved into the process of political decisions made in the EU. Therefore they created an E-Voting tool and the possibility to promote political aims in form of paneuropean initiatives.

Topics:
The Europeans are enlarging their Union.
The Europeans elect their President. Vote "Yes" or "No"

Drupal fits all, does it? Survey

This survey is being conducted to determine how much code Drupal
implementers actually need to write. The advent of modules such as
CCK, Views, Organic Groups, Workflow, Ubercart, E-Commerce, Panels and
Actions along with a dozen or so other modules has a empowered web
developers with the ability to create sites writing little or no
custom code. Is this true? If you are Drupal user please take the
survey at http://www.mahalasoft.co.za