Personal Democracy Plus Our premium content network. LEARN MORE You are not logged in. LOG IN NOW >

San Francisco Tells New York: Our Data Is Bigger Than Your Data

BY Sarah Lai Stirland | Monday, March 25 2013

Photo: Thomas Hawk/Flickr

San Francisco city officials have watched their brethren in New York have a day in the sun for a new emphasis on what you might call data-driven governance — and they're ready for their turn.

After all, San Francisco was the first city to pass a law asking city agencies to compile the data they keep about the city and to publish it online, in a machine-readable format that programmers could access and repurpose. New York and other cities, like Chicago, Philadelphia, and Madison, WI, merely followed suit, walking a trail blazed by then-Mayor Gavin Newsom with an executive order and then, in September 2010, with its open data law.

With that in mind, Mayor Ed Lee and Board of Supervisors President David Chiu are set on Thursday to make the case for a new San Francisco law that would establish a statutory role for the "chief data officer," someone that Lee is already looking to hire. That officer would be in charge of establishing a timeline for executing an open data plan, and liaise with the department data coordinators to communicate between the public and agency departments to prioritize the publication of high-value data sets, among many other responsibilities.

If it sounds familiar, that's because it's akin to how New York's new director of analytics, Michael Flowers, describes his role. Flowers told techPresident in February that he will be working across agencies to get more data ready to share, in part on the premise that the steps necessary to make the information in a department's spreadsheets and databases fit for public consumption will also make that information more useful to colleagues elsewhere within city government.

San Francisco's chief innovation officer, Jay Nath, says the legislation is informed by work going on in New York and elsewhere.

"We looked at [New York's] legislation very closely and leveraged a lot of the work that they had been doing and their thinking, but we think that we’ve kind of advanced open data even further than what New York City has done,” Nath said.

The layout in San Francisco will be a little different. (New York doesn't have a "chief data officer," for one.)

"This network of open data coordinators working with the chief data officer will really, in our minds, create that cohesive network that works to accelerate the release of data to the public, and create a new feedback loop," San Francisco's Chief Innovation Officer Jay Nath told techPresident.

San Francisco already maintains an open data portal called DataSF, but it's come in for criticism from open data advocates for not being updated frequently.

The legislation requires each agency to provide a catalogue of what information its officials possess, so that the conversation doesn't go around in circles, with city officials asking "What do you want," and the members of the public saying: "What do you have?" with city officials not being able to answer.

"And so our goal is to create a comprehensive list of all the data stores, and assets and spreadsheets that we as a city manage, and that are publicly available, so the public can indicate to us what they think should be released," Nath said. "And that’ll give us a sense of prioritization. We think that is a huge step forward."

Both New York and Philadephia's laws have their own versions of this requirement to inventory data.

The City by the Bay's proposed law is the fruit of its open data working group, which comprised city staff, summer interns from Harvard and Yale, staff from Code for America and the California city planning non-profit SPUR. The Sunlight Foundation's open data guidelines also provided a point of reference for the development of the law, Nath said.

Besides the chief data officer role, New York's law also doesn't have a couple of other interesting and thoughtful features of San Francisco's proposed rules: The attachment of Creative Commons-type licenses to the data, and a requirement specifying that going forward, city contracts with software vendors will expressly dictate that all data belongs to San Francisco city government, and not be held hostage by software vendors.

"We’ve had situations of that happening in San Francisco, where the contract between the city and the vendor was tilted towards the vendors’ benefit in terms of the data ownership," Nath said. "And our goal with the legislation is to ensure that our boilerplate language indicates that the default is owned by the city, and thus owned by the public."

The city's proposed rule also specifies that vendors will have to provide open application programming interfaces, or APIs, similar to New York's rules.

"[We will] ensure that whenever we buy new software, there is an API, a public API available, or some sort of equivalent, and the reason behind that is that we want to send a strong signal to the software makers who are selling to government, and specifically to San Francisco, that you need to ensure that the data is no longer locked, that it’s easily available to the public," Nath said. "And so hopefully, we’ll start to affect market change."

It might seem like a dry subject, but open data has powered many incredibly useful applications. For example, San Francisco recently partnered with Yelp to provide health ratings for restaurants. And Open City, a group of volunteer programmers in Chicago has created a handful of applications using Chicago's published open data, such as 2nd City Zoning, which "is an interactive map that lets you find out how your building is zoned, learn where to locate your business and explore zoning patterns throughout the city." Other apps mash up city to examine local economic trends, use school data to show parents what kind of a school district they live in, and building data to show building developments and demolitions, and violations.

News Briefs

RSS Feed today >

Another Co-Opted Hashtag: #MustSeeIran

The Twitter hashtag #MustSeeIran was created to showcase Iran's architecture, landscapes, and would-be tourist destinations. It was then co-opted by activists to bring attention to human rights abuses and infringements. Now Twitter is home to two starkly different portraits of a country. GO

At NETmundial Brazil: Is "Multistakeholderism" Good for the Internet?

Today and tomorrow Brazil is hosting NETmundial, a global multi-stakeholder meeting on the future of Internet governance. GO

Brazilian President Signs Internet Bill of Rights Into Law at NetMundial

Earlier today Brazil's President Dilma Rousseff sanctioned Marco Civil, also called the Internet bill of rights, during the global Internet governance event, NetMundial, in Brazil.


tuesday > Reboots As a Candidate Digital Toolkit That's a Bit Too Like launched with big ambitions and star appeal, hoping to crack the code on how to get millions of people to pool their political passions through their platform. When that ambition stalled, its founder Nathan Daschle--son of the former Senator--decided to pivot to offering political candidates an easy-to-use free web platform for organizing and fundraising. Now the new is out from stealth mode, entering a field already being served by competitors like NationBuilder, Salsa Labs and And strangely enough, seems to want its early users to ask for help. GO

Armenian Legislators: You Can Be As Anonymous on the 'Net As You Like—Until You Can't

A proposed bill in Armenia would make it illegal for media outlets to include defamatory remarks by anonymous or fake sources, and require sites to remove libelous comments within 12 hours unless they identify the author.


monday >

The Good Wife Looks for the Next Snowden and Outwits the NSA

Even as the real Edward Snowden faces questions over his motives in Russia, another side of his legacy played out for the over nine million viewers of last night's The Good Wife, which concluded its season long storyline exploring NSA surveillance. In the episode titled All Tapped Out, one young NSA worker's legal concerns lead him to becoming a whistle-blower, setting off a chain of events that allows the main character, lawyer Alicia Florrick (Julianna Margulies), and her husband, Illinois Governor Peter Florrick (Chris Noth), to turn the tables on the NSA using its own methods. GO

The Expanding Reach of China's Crowdsourced Environmental Monitoring Site, Danger Maps

Last week billionaire businessman Jack Ma, founder of the e-commerce company Alibaba, appealed to his “500 million-strong army” of consumers to help monitor water quality in China. Inexpensive testing kits sold through his company can be used to measure pH, phosphates, ammonia, and heavy metal levels, and then the data can be uploaded via smartphone to the environmental monitoring site Danger Maps. Although the initiative will push the Chinese authorities' tolerance for civic engagement and activism, Ethan Zuckerman has high hopes for “monitorial citizenship” in China.


The 13 Worst Bits of Russia's Current and Maybe Future Internet Legislation

It appears that Russia is on the brink of passing still more repressive Internet regulations. A new telecommunications bill that would require popular blogs—those with 3,000 or more visits a day—to join a government registry and conform to government-mandated standards is expected to pass this week. What follows is a list of the worst bits of both proposed and existing Russian Internet law. Let us know in the comments or on Twitter if we missed anything.


Transparency and Public Shaming: Pakistan Tackles Tax Evasion

In Pakistan, where only one in 200 citizens files their income tax return, authorities published a directory of taxpayers' details for the first time. Officials explained the decision as an attempt to shame defaulters into paying up.