Personal Democracy Plus Our premium content network. LEARN MORE You are not logged in. LOG IN NOW >

How to Evaluate the State of Open Data

BY David Eaves | Tuesday, May 8 2012

By tedeytan from Washington, DC (Library of Congress Reading Room Open House 7)

The Open Knowledge Foundation recently announced that it will organize and coordinate an Open Data Census. The intent is to create a basic baseline against which governments can measured around how much (and how relevant) their open data is.
If nothing else the census will be useful is forcing governments to reassess what “open” means. The OKFN has the very reasonable assessment that data is only open if it meets specific criteria, e.g. available in a machine-readable, publicly available, free of charge and openly licensed. (They also include available in a digital form, but one presumes that if it is machine-readable, it is digital). There are, in fact, a number of “open data” portals – particularly among local governments – that fail to meet these basic requirements. Seeing this fact reflected in the census may cause them to alter their licenses and formats.
More intriguing are the data sets the OKFN is opting to count. The details about the selection process are sparse, however. According to their announcement, data sets that will be counted were chosen based on “their breadth and relevance” and what “most governments could reasonably be expected to collect.” This list includes:

Election Results (national)
Company Register
National Map (Low resolution: 1:250,000 or better)
Government Budget (high level – spending by sector)
Government Budget (detailed – transactional level data)
Legislation (laws and statutes)
National Statistical Office Data (economic and demographic information)
National Postcode/ZIP database
Public Transport Timetables
Environmental Data on major sources of pollutants (e.g. location, emissions)

This is an excellent list. There is a solid balance between data that can encourage innovation and participation in democratic institutions and processes (election results & legislation) to data sets that can also help boost productivity across a variety of sectors (demographic and postal code data).
More challenging is understand the methodology of the census. There is a risk that the British biases – with its highly centralized national government – have strongly shaped the census. (OKFN is based in the UK.) Thus, while the census evaluates countries some of the data sets being counted are not controlled by national governments. For example – will national governments Canada or the United States get counted for public transport data if any of their cities release transit data? Indeed, transit data – while demonstrably useful – strikes me as an odd duck choice since it is almost always not managed by national governments. The same can be said for company/corporate registers, in which the most important data sets are managed by sub-national entities. Resolving – or at least making clear – these methodological issues will be important to foster a true apples to apples comparison.
There are, of course, a few data sets I would like to see added to the list. These include:
-Access to Information (ATIP or FOIA) made, completed, rejected and average response time, broken down by government entity.
-Government procurements and contracts, broken down by government entity
-Electoral Boundary Data
-Voting Booth Locations
-Land Registry Data
-Payments to Government for Extractive Industry Resources
-Foreign Aid Data
-Campaign Finance Data
-Registered Lobbyists List

As with the OKFN’s list, some of these data sets are targeted at reducing corruption or enhancing democracy. Others can both enhance democracy and reduce the friction costs in the economy. For example the land registry is important in both that knowing who owns what land is important for marginal groups as it enables them to gain access to capital, helps reduce the transaction costs around exchange land and, enables more transparent political discourse over who has influence, interests and resources in issues surrounding land ownership.
Finally, I would love to see another metric added to the census: the number of data sets that are shared in established schemas or standards. For example, a country that shares its foreign aid data in an open format should be congratulated. However, a country that shares its data in the International Aid Transparency Initiative data standard should receive a higher score still.
Personal Democracy Media is thankful to the Omidyar Network for its generous support of techPresident's WeGov section.

News Briefs

RSS Feed today >

Facebook Seeks Approval as Financial Service in Ireland. Is the Developing World Next?

On April 13 the Financial Times reported that Facebook is only weeks away from being approved as a financial service in Ireland. Is this foray into e-money motivated by Facebook's desire to conquer the developing world before other corporate Internet giants do? Maybe.


The Rise and Fall of Iran's “Blogestan”

The robust community of Iranian bloggers—sometimes nicknamed “Blogestan”—has shrunk since its heyday between 2002 – 2010. “Whither Blogestan,” a recent report from the University of Pennsylvania's Iran Media Program sought to find out how and why. The researchers performed a web crawling analysis of Blogestan, survey 165 Persian blog users, and conducted 20 interviews with influential bloggers in the Persian community. They found multiple causes of the decline in blogging, including increased social media use and interference from authorities.


tuesday >

Weekly Readings: What the Govt Wants to Know

A roundup of interesting reads and stories from around the web. GO

Russia to Treat Bloggers Like Mass Media Because "the F*cking Journalists Won't Stop Writing"

The worldwide debate over who is and who isn't a journalist has raged since digital media made it much easier for citizen journalists and other “amateurs” to compete with the big guys. In the United States, journalists are entitled to certain protections under the law, such as the right to confidential sources. As such, many argue that blogging should qualify as journalism because independent writers deserve the same legal protections as corporate employees. In Russia, however, earning a place equal to mass media means additional regulations and obligations, which some say will lead to the repression of free speech.


Politics for People: Demanding Transparent and Ethical Lobbying in the EU

Today the Alliance for Lobbying Transparency and Ethics Regulation (ALTER-EU) launched a campaign called Politics for People that asks candidates for the European Parliament to pledge to stand up to secretive industry lobbyists and to advocate for transparency. The Politics for People website connects voters with information about their MEP candidates and encourages them to reach out on Facebook, Twitter or by email to ask them to sign the pledge.


monday >

Security Agencies Given Full Access to Telecom Data Even Though "All Lebanese Can Not Be Suspects"

In late March, Lebanese government ministers granted security agencies unrestricted access to telecommunications data in spite of some ministers objections that it violates privacy rights. Global Voices reports that the policy violates Lebanon's existing surveillance and privacy law, Law 140, but has gotten little coverage from the country's mainstream media.


friday >

In Google Hangout, NYC Mayor de Blasio Talks Tech and Outer Borough Potential

New York City Mayor Bill de Blasio followed the lead of President Obama and New York City Council member Ben Kallos Friday by participating in a Google Hangout to help mark his first 100 days in office, in which the conversation focused on expanding access to technology opportunities through education and ensuring that the needs of the so-called "outer boroughs" aren't overlooked. GO

thursday >

In Pakistan, A Hypocritical Gov't Ignores Calls To End YouTube Ban

YouTube has been blocked in Pakistan by executive order since September 2012, after the “blasphemous” video Innocence of Muslims started riots in the Middle East. Since then, civil society organizations and Internet rights advocacy groups like Bolo Bhi and Bytes for All have been working to lift the ban. Last August the return of YouTube seemed imminent—the then-new IT Minister Anusha Rehman spoke optimistically and her party, which had won the majority a few months before, was said to be “seriously contemplating” ending the ban. And yet since then, Rehman and her party, the conservative Pakistan Muslim League (PML-N), have done everything in their power to maintain the status quo.


The #NotABugSplat Campaign Aims to Give Drone Operators Pause Before They Strike

In the #NotABugSplat campaign that launched this week, a group of American, French and Pakistani artists sought to raise awareness of the effects of drone strikes by placing a field-sized image of a young girl, orphaned when a drone strike killed her family, in a heavily targeted region of Pakistan’s Khyber-Pakhtunkhwa Province. Its giant size is visible to those who operate drone strikes as well as in satellite imagery. GO

Boston and Cambridge Move Towards More Open Data

The Boston City Council is now considering an ordinance which would require Boston city agencies and departments to make government data available online using open standards. Boston City Councilor At Large Michelle Wu, who introduced the legislation Wednesday, officially announced her proposal Monday, the same day Boston Mayor Martin Walsh issued an executive order establishing an open data policy under which all city departments are directed to publish appropriate data sets under established accessibility, API and format standards. GO

YouTube Still Blocked In Turkey, Even After Courts Rule It Violates Human Rights, Infringes on Free Speech

Reuters reports that even after a Turkish court ruled to lift the ban on YouTube, Turkey's telecommunications companies continue to block the video sharing site.