Personal Democracy Plus Our premium content network. LEARN MORE You are not logged in. LOG IN NOW >

How to Evaluate the State of Open Data

BY David Eaves | Tuesday, May 8 2012

By tedeytan from Washington, DC (Library of Congress Reading Room Open House 7)

The Open Knowledge Foundation recently announced that it will organize and coordinate an Open Data Census. The intent is to create a basic baseline against which governments can measured around how much (and how relevant) their open data is.
 
If nothing else the census will be useful is forcing governments to reassess what “open” means. The OKFN has the very reasonable assessment that data is only open if it meets specific criteria, e.g. available in a machine-readable, publicly available, free of charge and openly licensed. (They also include available in a digital form, but one presumes that if it is machine-readable, it is digital). There are, in fact, a number of “open data” portals – particularly among local governments – that fail to meet these basic requirements. Seeing this fact reflected in the census may cause them to alter their licenses and formats.
 
More intriguing are the data sets the OKFN is opting to count. The details about the selection process are sparse, however. According to their announcement, data sets that will be counted were chosen based on “their breadth and relevance” and what “most governments could reasonably be expected to collect.” This list includes:
 

Election Results (national)
Company Register
National Map (Low resolution: 1:250,000 or better)
Government Budget (high level – spending by sector)
Government Budget (detailed – transactional level data)
Legislation (laws and statutes)
National Statistical Office Data (economic and demographic information)
National Postcode/ZIP database
Public Transport Timetables
Environmental Data on major sources of pollutants (e.g. location, emissions)

 
This is an excellent list. There is a solid balance between data that can encourage innovation and participation in democratic institutions and processes (election results & legislation) to data sets that can also help boost productivity across a variety of sectors (demographic and postal code data).
 
More challenging is understand the methodology of the census. There is a risk that the British biases – with its highly centralized national government – have strongly shaped the census. (OKFN is based in the UK.) Thus, while the census evaluates countries some of the data sets being counted are not controlled by national governments. For example – will national governments Canada or the United States get counted for public transport data if any of their cities release transit data? Indeed, transit data – while demonstrably useful – strikes me as an odd duck choice since it is almost always not managed by national governments. The same can be said for company/corporate registers, in which the most important data sets are managed by sub-national entities. Resolving – or at least making clear – these methodological issues will be important to foster a true apples to apples comparison.
 
There are, of course, a few data sets I would like to see added to the list. These include:
 
-Access to Information (ATIP or FOIA) made, completed, rejected and average response time, broken down by government entity.
-Government procurements and contracts, broken down by government entity
-Electoral Boundary Data
-Voting Booth Locations
-Land Registry Data
-Payments to Government for Extractive Industry Resources
-Foreign Aid Data
-Campaign Finance Data
-Registered Lobbyists List

As with the OKFN’s list, some of these data sets are targeted at reducing corruption or enhancing democracy. Others can both enhance democracy and reduce the friction costs in the economy. For example the land registry is important in both that knowing who owns what land is important for marginal groups as it enables them to gain access to capital, helps reduce the transaction costs around exchange land and, enables more transparent political discourse over who has influence, interests and resources in issues surrounding land ownership.
 
Finally, I would love to see another metric added to the census: the number of data sets that are shared in established schemas or standards. For example, a country that shares its foreign aid data in an open format should be congratulated. However, a country that shares its data in the International Aid Transparency Initiative data standard should receive a higher score still.
 
Personal Democracy Media is thankful to the Omidyar Network for its generous support of techPresident's WeGov section.

News Briefs

RSS Feed today >

Organizing for Action Says It Can’t Move Climate Change Legislation In Congress

Beleaguered on one side by pressure to take a stand on the controversial Keystone XL pipeline and hoping, on the other, to hang on to the activist energy of the president's progressive base, Organizing for Action leadership and partners aren't holding out much hope for the power of grassroots organizing to motivate action in Congress. GO

Code for America Launches International Partnerships in Mexico, Germany and the Caribbean

Code for America today announced the launch of its first official international partnerships in Mexico, Germany and the Caribbean. After several years of recruiting technologists to spend a year in city halls across America, the organization will bring programmers and designers into close contact with governments in each of those three places to work on a specific problem area. GO

Top Russian Social Network VKontakte Briefly Banned "By Mistake"

The most popular social network in Russia worked its way onto a blacklist this Friday, allegedly “by mistake,” according to the state communications regulator. However, Pavel Durov, the founder of VKontakte, has had run-ins with the authorities in the past for allowing activists to organize protests on the platform. Some interpret this supposedly accidental blocking as a warning shot.

GO

thursday >

Anthony Weiner Launches NYC Mayoral Campaign Online With An Image of Pittsburgh

Former Congressman Anthony Weiner waxed lyrical about New York City in a YouTube video as he launched his bid to be the city's next mayor on Wednesday, but he did it against a backdrop that turned out to be the skyline for Pittsburgh, as a sharp-eyed observer on Twitter first noted Wednesday afternoon. GO

Revamped Data.gov Includes API Catalog

Federal officials are now offering a list of all APIs that have been released across the federal government as part of the Digital Government Strategy and a new data catalog that allows users to more easily search, sort and tag datasets, according to a post by Hyon Kim, deputy program director at the U.S. General Services Administration. With the announcements, the team behind Data.gov, a central public repository of machine-readable federal government data, is marking its fourth anniversary and the one year anniversary of the release of the Digital Government Strategy. GO

French Authorities Want to Tap (and Tax) Skype Calls

In spite of repeated requests from the French telecommunications authorities ARCEP, Skype has refused to classify itself as an electronic communications operator in France, which would require them to route emergency calls and allow the French police to intercept conversations. ARCEP has informed the Paris public prosecutor of Skype's refusal, and criminal charges might be brought against the company for failing to comply. This is yet another instance in recent months of France making things difficult for tech companies. Some worry that the overzealous government is discouraging technological progress in France, hindering business and economic growth.

GO

New Online Platform for Crowdsourced Videos About Human Rights Issues

Anyone with a phone and an Internet connection can be a citizen journalist, as was made clear in the hours and days after the Boston Marathon Bombings. Citizen journalism has its pros and cons, but it has popped up where most needed: after natural disasters or in war torn regions where career journalists might be barred. A new human rights initiative seeks to link citizen reporting in the form of online videos with mainstream media, governments and other policy makers. The online platform, called Irrepressible Voices, will both document human rights issues and work on solutions as a community.

GO

wednesday >

Facebook Becomes Full Member of Global Network Initiative

Facebook announced today that it has opted to become a full member of the Global Network Initiative, a group founded by Google, Microsoft and Yahoo to address the challenges technology companies face when dealing with governments about issues like freedom of expression and data privacy. GO

Russia's OGP Concerns Show That Transparency Matters

Last week, Russian officials announced they have withdrawn their letter of intent to join the Open Government Partnership. The Moscow Times has a statement to the Russian paper Kommersant from a presidential spokesman, saying, "We are not talking about winding up plans to join, but corrections in timing and the scale of participation are possible." So Russia may still be in. Just not soon. And maybe never. Confused? You're not alone. I actually find it fascinating that the Kremlin acts like "openness" and transparency matter. Here's why. GO

In Denmark, Online Tracking of Citizens is an Unwieldy Failure

Six years after Denmark passed a law mandating that telecommunication companies retain and store their customers' personal data for up to two years, local advocacy groups and the telecom industry are pushing for immediate changes to the legislation. The practice of keeping records of private citizens' Internet use is an unjustifiable invasion of privacy, they say. The police, meanwhile, have concluded that requiring telecoms to store subscriber data has not helped them track criminals, which was the the ostensible purpose of the practice. But the Danish government still wants to postpone an evaluation of the law for another two years. GO

"Accidental" Blocking of Australian Websites Raises Concerns About Government Censorship

An Australian government agency admitted last week to unintentionally blocking more than 1,200 perfectly legal websites in the process of shutting down one allegedly fraudulent site. In their defense, they pointed out that they have successfully blocked a number of websites in the past nine months without such digital collateral. This assertion came as no consolation to Australian netizens concerned about Internet censorship, especially opaque and hazily legal censorship.

GO

tuesday >

Honda Campaign Rolls Out Endorsements From Asian American Stars

Rep. Mike Honda (D-Calif.) rolled out several additional endorsements from Asian American leaders and celebrities Tuesday, with one of them vouching for his high-tech bona fides. GO

Here Are The People President Obama Hopes Will Repair American Elections

The Presidential Commission on Election Administration established by President Obama after problematic 2012 elections now has a web presence at SupporttheVoter.gov. Obama established the commission by executive order on March 28 "to identify best practices in election administration and to make recommendations to improve the voting experience." GO

After Oklahoma Disaster, Neighbors Look Online for Ways To Help

In echoes of the aftermath of Hurricane Sandy in the Northeast, social media sites and small business websites in and around tornado-wracked Moore, Okla., are full of offers of help, questions about missing pets and loved ones, and evidence that neighbors are willing to reach out to help one another in a disaster. On a single Facebook group, there's a Mexican restaurant in Oklahoma City promising free meals to first responders or people hit by the tornado; a mother a few hours' drive from Moore offering to open her door for children who might need a place to stay; a resident sharing a picture of a found dog and contact information for the owner to get in touch. GO

Change.org Lands $15 Million From Omidyar

Change.org capped an extraordinary few years of growth Tuesday with the announcement that it has landed a $15 million investment led by the Omidyar Network. GO

What German Politicians Think of Google Glass

The German government led by Chancellor Angela Merkel has not had the easiest relationship with Google. The company launched a public campaign against a law backed by her coalition that would require search engines to pay to show news articles in search results, with mixed results. What's more, Google has long had to navigate the privacy waters in Germany and throughout the European Union. But that has not stopped her federal minister for economics and technology, Philipp Rösler, from giving Google Glass an enthusiastic test run as he leads a delegation of German technology companies and politicians on a trip to Silicon Valley this week as part of German Valley Week. GO

Crowdsourcing Waste Management Solutions in Montenegro

For once we aren't talking about the worldwide scarcity of toilets, just good old-fashioned household waste. Montenegro has a garbage problem so bad even the tourists are complaining about it. A new mobile app sponsored by the Agency for Environmental Protection, NGO Ozon and United Nations Development Programme in Montenegro will hopefully get citizens involved in reporting illegal garbage dumps. GO

monday >

Her Majesty's Government Wants to Monetize Open Data

A new paper from the chair of the U.K. government's Open Strategy Board outlines the best practices for the government's open data policies. The government-commissioned Shakespeare Review – after author Stephan Shakespeare – looks into ways to monetize open data, and recommends an all-encompassing National Data Strategy.

GO

Will Silicon Valley "Disrupt" Politics With a Candidate for Congress?

Sean Parker, of Napster fame and now executive general partner at venture capital firm Founders Fund, has invested in political startups before. But last week, he went a step further — co-hosting a fundraising event for a candidate for Congress. Parker and SV Angel co-founder Ron Conway organized a crowd of Internet industry luminaries to support Ro Khanna, a former assistant deputy secretary in Barack Obama's Commerce Department. Khanna is preparing a challenge to Rep. Mike Honda (D-Calif.), whose newly redrawn congressional district encompasses Silicon Valley. GO

Burma's Upcoming Telecom Revolution Will Probably Not Bring Internet Freedom

Burma (Myanmar) is on the threshold of an Internet revolution, but Human Rights Watch has warned companies to proceed with caution or risk trampling Burmese citizens' rights. GO

More