Personal Democracy Plus Our premium content network. LEARN MORE You are not logged in. LOG IN NOW >

Open Data Goes Local with CA Data Camp and DataSF

BY Sarah Granger | Friday, November 13 2009

Almost three months ago, the City and County of San Francisco launched a site called DataSF where they publish data sets from a variety of city departments for public consumption and application development. The initiative, led by Jay Nath in the Department of Technology, was inspired by President Obama's transparency directive on his first day in office. They then looked at what had been done with Apps for Democracy in Washington, D.C. and what was presented at through Vivek Kundra's office. While we weren't sure what to make ot if at first, San Francisco's history of innovation and progressive policies left us hopeful that this program could bring good things.

Meanwhile, local organizers decided it was time to put together a barcamp and apps contest that took into account all of the people involved in open government projects around the Bay Area. That happened in SF this past Saturday at Citizen Space. Organizers included David Cohn of Spot.Us, Susan Mernit of California Watch and others. Sponsors, most of whom who made cameos, included The Sunlight Foundation, Gov 2.0 (O'Reilly Media), Craig Newmark,, mobclix, haku wale, DataSF and Mitch Kapor. In other words, the local open gov't heavy hitters agreed this was a good idea. (Note: Twitter stream can be found by searching #cadata.)

We spent the first hour mingling, snacking and stacking the board for sessions; the space allowed for three at a time, split into four segments, two before lunch and two after. (See a great blog account of the entire day here.) Later, in another room nearby, an apps contest got started. Topics generally related back to data and local, including transportation data, financial data, stimulus data, healthcare data, etc. We also had a session led by Adriel Hampton of Gov 2.0 radio, focused on how it all ties into advocacy online, and since there was a heavy concentration of media people there, we talked in a session I facilitated about how the media can use data, crowdsourcing data, and good data sources. The visualization of data session was interesting too. That one included a lot of link sharing, collected by Adina Levin on her delicious feed. As one would expect, there was a lot of talk about APIs and building data sets.

The bulk of the discussion, of course, led back to DataSF. We were lucky Nath and some of his colleagues were there to give us details on what's happening with the initiative, and it was clear they wanted to hear from the community. So far, they have learned a lot since launching nearly three months back and they're clearly responding to public input. On October 21st, while Tim O'Reilly was in SF for the Web 2.0 Summit, Mayor Newsom and other city officials announced an Open Data Executive Directive requiring all city departments to make non confidential datasets available - a big step. Nath and his group developed their site on a platform inexpensive enough that smaller cities could use it as well, so they could provide a proving grounds for this type of work.

Where they get their input is through the 'Suggest Other Datasets' page. Nath explains that "people find interest in a lot of different areas" so this helps them sift through where the greatest interest lies. City residents can describe what they would like to see and the community votes for the best ideas. (The City can't do everything - too many resource constraints.) However, they haven't gotten the traffic that might be expected. So far, 25,000 have visited the site, but only 300-400 have voted. It could be that the voting page isn't prevalent enough on the site or that enough people just don't know it's there or how to use it, but I expect that will grow.

General problems dealing with government bureaucracy and data latency arise as is to be expected. Nath says cleaning up data can be difficult, the data hasn't all been standardized, some of it hasn't even been gathered (such as parking data). And some department heads fear the exposure of their data will somehow expose flaws in how they manage. "A lot of it is a culture shift and it takes time," Nath said.

DataSF also has an App Showcase that already highlights some of the applications developed over the past few months. Hopefully some of the apps from the Apps Contest will be up there soon. And they post an Agency Scorecard to incentivize departments to provide their datasets. Nath has reached out to other open government leaders in cities and agencies around the country, but he hasn't found as much collaboration as he would like. He has even offered to share all of the policies, procedures, legal documents and related information their department has and is developing during this project - something just as valuable as the data itself. So far, no one has bitten. However, he says the Sunlight Foundation and other national groups have been helpful.

At the 20,000 foot level, I think this event provided a good indicator of where we are in the online transparency movement. We have great people and great ideas, a lot of smart developers and the beginnings of sensible policy. What we don't have is 100% buy-in from people within government or complete understanding in the public of what this all means. As San Francisco tends to be an early adopter, we have a long way to go before all other major cities play their part, but we're lucky they are willing to share what they're learning. I highly encourage others considering opening up government data to reach out and talk to them about their experiences to date. (Find Nath at @Jay_Nath on Twitter.)

As to the CA Data Camp - I get the sense that this will not be the last time we convene. I look forward to seeing where these initiatives lead over the next year and I expect the community will rally in support of continued engagement.

News Briefs

RSS Feed wednesday >

Another Co-Opted Hashtag: #MustSeeIran

The Twitter hashtag #MustSeeIran was created to showcase Iran's architecture, landscapes, and would-be tourist destinations. It was then co-opted by activists to bring attention to human rights abuses and infringements. Now Twitter is home to two starkly different portraits of a country. GO

What Has the EU Ever Done For Us?: Countering Euroskepticism with Viral Videos and Monty Python

Ahead of the May 25 European Elections, the most intense campaigning may not be by the candidates or the political parties. Instead, some of the most passionate campaigns are more grassroots efforts focused on for a start stirring up the interest of the European electorate. GO

At NETmundial Brazil: Is "Multistakeholderism" Good for the Internet?

Today and tomorrow Brazil is hosting NETmundial, a global multi-stakeholder meeting on the future of Internet governance. GO

Brazilian President Signs Internet Bill of Rights Into Law at NetMundial

Earlier today Brazil's President Dilma Rousseff sanctioned Marco Civil, also called the Internet bill of rights, during the global Internet governance event, NetMundial, in Brazil.


tuesday > Reboots As a Candidate Digital Toolkit That's a Bit Too Like launched with big ambitions and star appeal, hoping to crack the code on how to get millions of people to pool their political passions through their platform. When that ambition stalled, its founder Nathan Daschle--son of the former Senator--decided to pivot to offering political candidates an easy-to-use free web platform for organizing and fundraising. Now the new is out from stealth mode, entering a field already being served by competitors like NationBuilder, Salsa Labs and And strangely enough, seems to want its early users to ask for help. GO

Armenian Legislators: You Can Be As Anonymous on the 'Net As You Like—Until You Can't

A proposed bill in Armenia would make it illegal for media outlets to include defamatory remarks by anonymous or fake sources, and require sites to remove libelous comments within 12 hours unless they identify the author.


monday >

The Good Wife Looks for the Next Snowden and Outwits the NSA

Even as the real Edward Snowden faces questions over his motives in Russia, another side of his legacy played out for the over nine million viewers of last night's The Good Wife, which concluded its season long storyline exploring NSA surveillance. In the episode titled All Tapped Out, one young NSA worker's legal concerns lead him to becoming a whistle-blower, setting off a chain of events that allows the main character, lawyer Alicia Florrick (Julianna Margulies), and her husband, Illinois Governor Peter Florrick (Chris Noth), to turn the tables on the NSA using its own methods. GO

The Expanding Reach of China's Crowdsourced Environmental Monitoring Site, Danger Maps

Last week billionaire businessman Jack Ma, founder of the e-commerce company Alibaba, appealed to his “500 million-strong army” of consumers to help monitor water quality in China. Inexpensive testing kits sold through his company can be used to measure pH, phosphates, ammonia, and heavy metal levels, and then the data can be uploaded via smartphone to the environmental monitoring site Danger Maps. Although the initiative will push the Chinese authorities' tolerance for civic engagement and activism, Ethan Zuckerman has high hopes for “monitorial citizenship” in China.


The 13 Worst Bits of Russia's Current and Maybe Future Internet Legislation

It appears that Russia is on the brink of passing still more repressive Internet regulations. A new telecommunications bill that would require popular blogs—those with 3,000 or more visits a day—to join a government registry and conform to government-mandated standards is expected to pass this week. What follows is a list of the worst bits of both proposed and existing Russian Internet law. Let us know in the comments or on Twitter if we missed anything.


Transparency and Public Shaming: Pakistan Tackles Tax Evasion

In Pakistan, where only one in 200 citizens files their income tax return, authorities published a directory of taxpayers' details for the first time. Officials explained the decision as an attempt to shame defaulters into paying up.