With Open Data, The Transparency Medium Can Matter As Much as the Message
BY David Eaves | Wednesday, January 30 2013
This is going to sound crazy, but bear with me: Transparency matters, even when no one seems to be watching.
If you've not read the IBM Center for The Business of Government report Recovery Act Transparency: Learning from States' Experience — but, let's be honest, who hasn't? — then check it out. Then, go read Exposing Corrupt Politicians: The Effects of Brazil's Publicly Released Audits on Electoral Outcomes by Claudio Ferraz and Frederico Finan. Not only do both make for fascinating — and ultimately uplifting — readings, they shed some light on differing strategies, as well as metrics for success, for anti-corruption efforts.
The somewhat sobering conclusion of the Recovery Act Transparency report is that very few people downloaded or made use of data from Recovery.gov, the website where officials published information on Recovery Act spending. And yet, despite this fact, the number of problems with the funds was very low. As the studies' authors note, the Recovery Act "sustained an exceptional level of spending disclosure compliance and low rates of fraud, waste, and abuse of funds."
I've always felt that one of the worse metrics of success for most transparency initiatives was the number of downloads. I would love to believe that Americans should have been more interested in how and where the money of an $800 billion "new New Deal" was being spent. I'm surprised there was no Tea Party-led effort to use the data from Recovery.gov to track down every mis-spent or poorly allocated dollar. But my experience is that attracting people to look at information requires their either be a problem to solve or a strong narrative. When people started sifting through British MP expense claims, for example, it was because there was already evidence of wrongdoing and the "problem" was understanding the breadth of the issue.
And there were ample opportunities to discover problems. Recovery.gov data had several high-profile users, like ProPublica. That said, efforts to attract a "crowd" were generally unsuccessful, mostly because, the report suggests, there was little fraud to discover.
This is a good outcome. Hoping lots of people will police a problem should always be the mechanism of last resort — relying on "the crowd" suggests that institutions or authorized oversight bodies can't handle their responsibilities or, at the least, do not look like they can. What makes Recovery.gov interesting to me is that the threat of a crowd may have still created all the right incentives for the institutions and actors within the system.
Here is the sobering way the authors describe it:
While we cannot definitively establish that transparency-related, anticipatory mechanisms resulted in low rates of fraud in the Recovery Act program, it appears that the internal institutional effort to combine predictive modeling, risk analysis, "shaming," and anticipatory deterrence may have worked to drive accountability and keep fraud in check.
Note the predictive modeling. This reference is particularly interesting and invokes one of the great orthogonal benefits of open data. Open data compels governments to get more savvy in analyzing their data because as helpful as it is to have a watchdog organization or engaged citizen uncover a systemic problem it nonetheless inevitably leads to the question... "why in the hell didn't you catch that." Indeed, the authors noted that this was a major innovation of the Recovery Act:
"The Recovery Operations Center was a control room with a wall of computer screens situated among the Recovery Board's otherwise nondescript offices in downtown Washington D.C. Inside, analysts used sophisticated predictive modeling and risk analysis techniques on large sets of data culled from recipient reports and other government databases to search for the likelihood of fraud by Recovery Act-funded projects. 'The idea: shift the focus from detecting fraud to preventing and interrupting fraud.' "
As I mentioned above, large numbers of page views can be an interesting way to identify problems. But so too can be the potential for large numbers of page views. I think this also explains why the cities doing the most interesting analytics work to find efficiencies and improve performance tend to also be those most aggressive about open data. As great as it might be to have citizens use your data to find an inefficiency, getting scooped by citizens is still not as useful to a government as being seen to find and fix a problem itself.
The other amazing finding of the recovery report is how effectively the Recovery Act's transparency requirements became institutionalized in some states and at the federal level. It may be that one of the most important and lasting impacts of the Recovery Act is not the physical infrastructure it leaves behind, but how it has changed the infrastructure of governance. Now, a number of states — as well as the federal government — have a vastly improved capacity to disclose where money flows and to be transparent. There is an implicit benefit to citizens who could, in theory, monitor governments more closely. But one of the biggest benefactors is government, which will be better able to track where it is spending money, making it easier to identify not just fraud, but duplication. Indeed, the authors note that "state officials were principal users of Recovery Act data as it allowed them to manage and track federal spending in near-real time."
If more transparency means better management and oversight by government, this is not a bad outcome, even if it means there are few downloads or page views.
I'm not saying that page views are unimportant, only that I don't believe they are always among the most important metrics for success. Nor am I saying that recovery.gov was perfect in every way. For example, it failed to deliver on its promise to show spending down to the block or project level, and this may have hampered citizens, reporters and others to "fact check" the work being done. But it did seem to foster a system of checks and balances that made fraud and waste more difficult.
Ferraz and Finan's paper Exposing Corrupt Politicians: The Effects of Brazil's Publicly Released Audits on Electoral Outcomes offers a fantastic example of when public consumption of data does matter.
Ferraz and Finan explore the impact of a 2003 law which saw the Brazilian federal government audit roughly 60 randomly chosen municipalities every year. The results of these audits are then distributed publicly. The authors could then look at the electoral performance of mayors that were audited and compare them against those who were not.
The results were exceedingly encouraging. The performance of audited and non-audited mayors were identical, with one exception. If the audit determined there were elevated levels of corruption, incumbent mayors did much worse. In the authors' own words:
"For every additional corrupt violation reported, the audit policy reduced the incumbent's likelihood of re-election by approximately 20 percent. The effect of the policy was similar for other measures of electoral performance, such as the change in vote share and margin of victory. These results suggest that voters not only care about corruption, but once empowered with the information, update their prior beliefs and punish corrupt politicians at the polls."
The audits did not just punish the corrupt — they also rewarded the virtuous. Politicians that were believed to be corrupt but then cleared by the audit performed well in polls.
More interestingly, the dissemination (similar to page views) mattered. A lot. In municipalities with local radio stations, the results were worse still for corrupt mayors that had been audited.
What I think is encouraging is that page views (or dissemination) is not always necessary to improve transparency — the mere act of forcing a system to be more transparent can shift the behaviour of the actors in the system in positive ways. That said, transparency can empower the public if other checks on corruption fail to prevent it.
All this is to say that designers of transparency rules should think carefully about their goals when drafting policies. Public attention can be difficult to harness so designing a system that leverages it needs to be finely tuned — although the benefits can be clear.
As more governments — like the Italian government, which just launched the OpenCoesione website — begin to share more data about where granting dollars flow, there are lessons to be garnered and, very likely, more comparative research opportunities.
Personal Democracy Media is grateful to The Omidyar Network for its generous support of techPresident's WeGov section.