New Hampshire Opens its State's Legislative Data
BY Nick Judd | Tuesday, February 1 2011
Chalk up a new one for the open-government-data geeks.
In the past few days, New Hampshire's General Court, as the state legislature is officially known, started releasing data on legislation and legislators in nerd-friendly, "pipe-separated" files, uploaded daily. In non-geek speak, this means the data is presented in a way that any competent web developer can easily process for use in an application or a researcher can feed into a database system to explore.
The General Court has an old-fashioned legislative lookup interface that allows, for example, for searches to find individual bills — but that's not useful for people who want to track legislation by topic area, or remix information on bills in the legislature in any other way.
A New Hampshire political group, the New Hampshire Liberty Alliance, had been using a scraper to pull that data off the state's website and republish it, George Lambert, a first-term Republican representing the Hillsborough 27 district in the state House and a selectman in his town of Litchfield, explained to me Monday. Lambert, for those of you wondering how a legislator knows what a scraper is, is only a part-time legislator; his full-time career is as a software developer.
"They had to screen scrape between six to 8,000 pages every day," he said. "And it was causing [the state legislature's IT staff] a lot of inconvenience to have a lot of people going in there and trying to analyze the information."
This was a practical financial point that open government advocates were able to make to further the cause of transparency, said Seth Cohn, a Drupal developer who just began his first term representing the Merrimack 6 district in the state and lives in Canterbury.
"State IT people realized that not only were they doing the right thing but they were also reducing the server loads," Cohn, who also ran as a Republican, said.
James Turk, the Sunlight Foundation developer who is leading the data side of a partnership with the Participatory Politics Foundation to clean and publish information about the workings of state legislatures in an easily searchable and shareable way, said the mass scraping of information from other state websites is causing similar stress to infrastructures nationwide.
"As far as the data itself," he said, "it looks very useful and will make it a lot easier for a project like ours to take the NH data in without any possibility of accidental errors or creating excessive load on their website."
This is just the beginning of the state legislature's open-data efforts, said the state House's policy director, George Moore. The staff is working to publish data on roll call votes, which are not currently available, he said.
"Obviously, this is a baby step," he said.
This isn't the only example of increased geekitude in the New Hampshire state legislature — come back soon for more updates from that state.