The closing down of an effort known as Expert Labs this month acted as a marker of sorts in the open government movement. Epitomizing the general ethos of the time, here was a group of Internet-famous hipster technologists and personalities who had decided to storm the barricades and focus their collective attention on helping the federal government to break out of the Beltway bubble to connect better with the public when making policy decisions. Why shouldn't the world have been excited about what kind of change they could potentially have brought about?

TechPresident alumnus Nancy Scola wondered at the time in 2010 when Expert Labs was first getting going: "Can what we know in 2010 about the wisdom of crowds really improve policy making? It's an intriguing idea, but the devil is in the details."

Two years and several reports later, we thought we'd try to look at how Expert Labs fared. The premise behind the project was that the federal government could and should engage in conversations with people on their existing social networks. The idea was to use existing commercial social networks to crowdsource policy decisions and to synthesize the responses in an intelligent manner. The tool that the team built, ThinkUp, was a free, open-source web application that needed to be downloaded onto a server. Once installed, it allowed its users to download all of their social media activity into a database that the user controlled. (The application is going to live on and be turned into the basis for a private company.)

At the time, Expert Labs' Founding Director Anil Dash said that the inspiration behind the project was, in part, New York Law School Beth Noveck's Peer-to-Patent project. That pioneering experiment sought to improve the patent issuance process by inviting the public to help the patent office reduce its backlog, in particular by assisting with the search for "prior art" that might invalidate a patent application. The idea was that the public could provide more and better input than the existing search procedures undertaken by patent examiners.

Could similar policy feedback mechanisms be built across the federal government through existing social networks? That was an experiment that Expert Labs undertook with ThinkUp, which was backed by a grant from the MacArthur Foundation and support from the American Association for the Advancement of Science. Behind it also were others who had been successful in the private sector, and within the Beltway itself: Gina Trapani was Expert Labs' lead developer, Andy Baio was its data publisher and Clay Johnson, was its DC policy expert and liaison with federal agencies.

Two years later, ThinkUp's impact on the way government agencies formulate policy and includes the wider public in the process has been minimal, it appears. To wit: last December, Expert Labs established its Federal Social Media Index based on data aggregated by ThinkUp in an effort to highlight successful interactions between government agencies and the public. But a glance at the latest report shows that while agencies are asking questions using online social media tools like Twitter, the public doesn't seem to be particularly interested in answering those questions on their existing social networks.

For the week of March 19 to 25, for example, the 125 agencies tracked by Expert Labs asked 343 questions, but the index reports that only three people responded. The quality of the questions asked and answers recieved is hardly awe-inspiring.
The longer list of agencies with a tally of questions asked and answered shows that most agencies (if they asked a question at all) did not get any responses from the public.

"We do agree agencies aren't doing the best job of engaging on these networks yet," wrote Dash in an e-mail to techPresident in response to some questions about lessons learned from the Expert Labs experience. "One key finding we've focused on in our final reports is that the division between communications/outreach arms of agencies, which typically manage social networking accounts, and the policy making groups within agencies, which actually impact the decisions being made, is a pretty significant barrier to public participation."

But Jed Sundwall, president of Measured Voice, and a social media consultant for several federal agencies, believes the problem facing ThinkUp and its successful deployment and use among those agencies boils down to a combination of the mundane and philosophical. Sundwall has downloaded and experimented with ThinkUp himself.

On the mundane logistical side, federal IT departments are risk averse and don't want to install anything that might pose a potential security risk on their web servers, he said.

"For most agencies, IT staff exist to make sure the department's e-mail system works," he said. "Few agencies have the resources to work with something like ThinkUp."

Then there's the broader issue of the assumptions behind ThinkUp's approach to citizen engagement and crowd-sourcing policy.

"Gathering useful input is a massive design challenge," Sundwall notes. "Agencies can't just ask on Twitter: 'Hey what do you guys think about fracking?'"

Instead, getting useful public feedback involves a lot of strategic planning and reaching out to specific audiences, involving "a lot of hard work that software can't solve," he said.

Some of that work involves posing the right questions, figuring out what the agency is really asking of the public, and making sure they're reaching out to the right pool of people who would be interested in providing feedback on a specific policy issue.

"Mostly, people just want government to work, and the only time they want to get involved is when it's not working," he asserts. "I disagree with the idea that people want to chat with the government: If government agencies do engage, they shouldn't waste people's time, and it should be purposeful."

Expert Labs' Clay Johnson has recommended that agencies should reach out to the public on channels that are more familiar to them than through the obscure web forms that appear on agency web sites, or simply by publishing their existing rule-making procedures through the centralized web site Regulations.gov. He suggests that one way of doing this is through tighter integration of agencies' rule-making tools with social media so that citizens can submit their comments and ideas through those channels.

The problem with this approach, he acknowledged in his blog post on the subject, is that federal law in the form of the Administrative Procedure Act would limit the admissibility of comments submitted through social media channels. The law requires agencies to be able to verify the identity of commenters, and to be able to follow up if agency staffers want a clarification.

But the existence of this rule might have also stymied the evolution of ThinkUp as a viable platform to crowd-source public input into policy-making process. Asked about this, and whether Expert Labs had made any progress in prodding the federal government to provide some clarity on the issue, Dash wrote: "We have definitely heard about some good progress being made on this front, both from people in the executive branch and from interested parties on the legislative side. Since those conversations are still ongoing, I'll probably defer on more details until a time if or when there's an announcement, but I remain very optimistic about the possibility overall, though of course it's frustrating having to be patient about it!"