You are not logged in. LOG IN NOW >

Government Shutdown Sets Off Data and API Scramble

BY Miranda Neubauer | Thursday, October 3 2013

Among the many casualties of the government shutdown are the websites and data sources that researchers, civic hackers and others use on a regular basis for a variety of online applications, visualization projects and studies.

The disappearance of resources like data.gov and census.gov has forced those relying on the data to act quickly to find creative solutions or work together to gather backed-up information.

Michal Migurski, CTO for Code for America, has been helping to coordinate a back-up of census data files on a website hosted by the OpenStreetMap US Foundation via the Oregon State Open Source Lab.

"The impact has been just colossal," Migurski said. He has been working to gather datasets from others who have been backing them up as part of their own work or kept copies for their own use. While his focus so far has been on census data, he said the site would also have the capacity to host other kinds of datasets

Darrell Fuhriman, engineering manager at Renewable Funding, has made available census shapefiles and summary files for 2010 and 2011 that provide information on demographics for the entire country. Data artist Eric Fischer has made available recent years of so-called TIGER data which helps users create maps of U.S. states and counties, illustrate demographic trends and characteristics of local neighborhoods.

Migurski praised the government agencies for having made that data "aggressively available" before the shutdown so that it was easy for users to mirror, copy and download and then make available "at the last minute."

In the future, in advance of any similar occasion, he said he expected more proactive backup efforts to take place, adding that he himself would likely download several datasets for that purpose as soon as as the census servers return.

"For Code for America, it's really unfortunate that this is happening," he said. But he added that the shutdown "makes very clear exactly what it is the federal government is doing for us," including offering the high quality data resources many users begin to take for granted. "It shows all the different ways that government affects our lives."

Another service affected is voter-registration tool TurboVote.

"In order to match people to their local election offices, we rely on an API offered by the FCC that converts geographic coordinates to counties," TurboVote Co-Founder Kathryn Peters wrote in an e-mail. "While Google can do the same thing, the FCC API also provides a FIPS code (a unique ID for every county), which is much easier for us to use than a county name/state combination. We had to scramble to substitute another process for matching addresses to county FIPS codes so our new voters would receive correct information about their local election officials."

Startup Exversion also has some Data.gov resources available.

In a blog post yesterday, Eric Mill from the Sunlight Foundation wrote that the data and API shutdown illustrated the need for government agencies to publish downloadable bulk data at the same time as they publish APIs, to encourage the reuse and republication of that data, to document what data will be available in the event of a shutdown and to offer links to alternative sources that are accessible during a shutdown.

The Pew Research Center and the Ocean Conservancy also highlighted other "data casualties" of the shutdown, while Search Engine Land examined what the shutdown of government websites means for their standing in search engine results.