You are not logged in. LOG IN NOW >

San Francisco Tells New York: Our Data Is Bigger Than Your Data

BY Sarah Lai Stirland | Monday, March 25 2013

Photo: Thomas Hawk/Flickr

San Francisco city officials have watched their brethren in New York have a day in the sun for a new emphasis on what you might call data-driven governance — and they're ready for their turn.

After all, San Francisco was the first city to pass a law asking city agencies to compile the data they keep about the city and to publish it online, in a machine-readable format that programmers could access and repurpose. New York and other cities, like Chicago, Philadelphia, and Madison, WI, merely followed suit, walking a trail blazed by then-Mayor Gavin Newsom with an executive order and then, in September 2010, with its open data law.

With that in mind, Mayor Ed Lee and Board of Supervisors President David Chiu are set on Thursday to make the case for a new San Francisco law that would establish a statutory role for the "chief data officer," someone that Lee is already looking to hire. That officer would be in charge of establishing a timeline for executing an open data plan, and liaise with the department data coordinators to communicate between the public and agency departments to prioritize the publication of high-value data sets, among many other responsibilities.

If it sounds familiar, that's because it's akin to how New York's new director of analytics, Michael Flowers, describes his role. Flowers told techPresident in February that he will be working across agencies to get more data ready to share, in part on the premise that the steps necessary to make the information in a department's spreadsheets and databases fit for public consumption will also make that information more useful to colleagues elsewhere within city government.

San Francisco's chief innovation officer, Jay Nath, says the legislation is informed by work going on in New York and elsewhere.

"We looked at [New York's] legislation very closely and leveraged a lot of the work that they had been doing and their thinking, but we think that we’ve kind of advanced open data even further than what New York City has done,” Nath said.

The layout in San Francisco will be a little different. (New York doesn't have a "chief data officer," for one.)

"This network of open data coordinators working with the chief data officer will really, in our minds, create that cohesive network that works to accelerate the release of data to the public, and create a new feedback loop," San Francisco's Chief Innovation Officer Jay Nath told techPresident.

San Francisco already maintains an open data portal called DataSF, but it's come in for criticism from open data advocates for not being updated frequently.

The legislation requires each agency to provide a catalogue of what information its officials possess, so that the conversation doesn't go around in circles, with city officials asking "What do you want," and the members of the public saying: "What do you have?" with city officials not being able to answer.

"And so our goal is to create a comprehensive list of all the data stores, and assets and spreadsheets that we as a city manage, and that are publicly available, so the public can indicate to us what they think should be released," Nath said. "And that’ll give us a sense of prioritization. We think that is a huge step forward."

Both New York and Philadephia's laws have their own versions of this requirement to inventory data.

The City by the Bay's proposed law is the fruit of its open data working group, which comprised city staff, summer interns from Harvard and Yale, staff from Code for America and the California city planning non-profit SPUR. The Sunlight Foundation's open data guidelines also provided a point of reference for the development of the law, Nath said.

Besides the chief data officer role, New York's law also doesn't have a couple of other interesting and thoughtful features of San Francisco's proposed rules: The attachment of Creative Commons-type licenses to the data, and a requirement specifying that going forward, city contracts with software vendors will expressly dictate that all data belongs to San Francisco city government, and not be held hostage by software vendors.

"We’ve had situations of that happening in San Francisco, where the contract between the city and the vendor was tilted towards the vendors’ benefit in terms of the data ownership," Nath said. "And our goal with the legislation is to ensure that our boilerplate language indicates that the default is owned by the city, and thus owned by the public."

The city's proposed rule also specifies that vendors will have to provide open application programming interfaces, or APIs, similar to New York's rules.

"[We will] ensure that whenever we buy new software, there is an API, a public API available, or some sort of equivalent, and the reason behind that is that we want to send a strong signal to the software makers who are selling to government, and specifically to San Francisco, that you need to ensure that the data is no longer locked, that it’s easily available to the public," Nath said. "And so hopefully, we’ll start to affect market change."

It might seem like a dry subject, but open data has powered many incredibly useful applications. For example, San Francisco recently partnered with Yelp to provide health ratings for restaurants. And Open City, a group of volunteer programmers in Chicago has created a handful of applications using Chicago's published open data, such as 2nd City Zoning, which "is an interactive map that lets you find out how your building is zoned, learn where to locate your business and explore zoning patterns throughout the city." Other apps mash up city to examine local economic trends, use school data to show parents what kind of a school district they live in, and building data to show building developments and demolitions, and violations.