Personal Democracy Plus Our premium content network. LEARN MORE You are not logged in. LOG IN NOW >

San Francisco Tells New York: Our Data Is Bigger Than Your Data

BY Sarah Lai Stirland | Monday, March 25 2013

Photo: Thomas Hawk/Flickr

San Francisco city officials have watched their brethren in New York have a day in the sun for a new emphasis on what you might call data-driven governance — and they're ready for their turn.

After all, San Francisco was the first city to pass a law asking city agencies to compile the data they keep about the city and to publish it online, in a machine-readable format that programmers could access and repurpose. New York and other cities, like Chicago, Philadelphia, and Madison, WI, merely followed suit, walking a trail blazed by then-Mayor Gavin Newsom with an executive order and then, in September 2010, with its open data law.

With that in mind, Mayor Ed Lee and Board of Supervisors President David Chiu are set on Thursday to make the case for a new San Francisco law that would establish a statutory role for the "chief data officer," someone that Lee is already looking to hire. That officer would be in charge of establishing a timeline for executing an open data plan, and liaise with the department data coordinators to communicate between the public and agency departments to prioritize the publication of high-value data sets, among many other responsibilities.

If it sounds familiar, that's because it's akin to how New York's new director of analytics, Michael Flowers, describes his role. Flowers told techPresident in February that he will be working across agencies to get more data ready to share, in part on the premise that the steps necessary to make the information in a department's spreadsheets and databases fit for public consumption will also make that information more useful to colleagues elsewhere within city government.

San Francisco's chief innovation officer, Jay Nath, says the legislation is informed by work going on in New York and elsewhere.

"We looked at [New York's] legislation very closely and leveraged a lot of the work that they had been doing and their thinking, but we think that we’ve kind of advanced open data even further than what New York City has done,” Nath said.

The layout in San Francisco will be a little different. (New York doesn't have a "chief data officer," for one.)

"This network of open data coordinators working with the chief data officer will really, in our minds, create that cohesive network that works to accelerate the release of data to the public, and create a new feedback loop," San Francisco's Chief Innovation Officer Jay Nath told techPresident.

San Francisco already maintains an open data portal called DataSF, but it's come in for criticism from open data advocates for not being updated frequently.

The legislation requires each agency to provide a catalogue of what information its officials possess, so that the conversation doesn't go around in circles, with city officials asking "What do you want," and the members of the public saying: "What do you have?" with city officials not being able to answer.

"And so our goal is to create a comprehensive list of all the data stores, and assets and spreadsheets that we as a city manage, and that are publicly available, so the public can indicate to us what they think should be released," Nath said. "And that’ll give us a sense of prioritization. We think that is a huge step forward."

Both New York and Philadephia's laws have their own versions of this requirement to inventory data.

The City by the Bay's proposed law is the fruit of its open data working group, which comprised city staff, summer interns from Harvard and Yale, staff from Code for America and the California city planning non-profit SPUR. The Sunlight Foundation's open data guidelines also provided a point of reference for the development of the law, Nath said.

Besides the chief data officer role, New York's law also doesn't have a couple of other interesting and thoughtful features of San Francisco's proposed rules: The attachment of Creative Commons-type licenses to the data, and a requirement specifying that going forward, city contracts with software vendors will expressly dictate that all data belongs to San Francisco city government, and not be held hostage by software vendors.

"We’ve had situations of that happening in San Francisco, where the contract between the city and the vendor was tilted towards the vendors’ benefit in terms of the data ownership," Nath said. "And our goal with the legislation is to ensure that our boilerplate language indicates that the default is owned by the city, and thus owned by the public."

The city's proposed rule also specifies that vendors will have to provide open application programming interfaces, or APIs, similar to New York's rules.

"[We will] ensure that whenever we buy new software, there is an API, a public API available, or some sort of equivalent, and the reason behind that is that we want to send a strong signal to the software makers who are selling to government, and specifically to San Francisco, that you need to ensure that the data is no longer locked, that it’s easily available to the public," Nath said. "And so hopefully, we’ll start to affect market change."

It might seem like a dry subject, but open data has powered many incredibly useful applications. For example, San Francisco recently partnered with Yelp to provide health ratings for restaurants. And Open City, a group of volunteer programmers in Chicago has created a handful of applications using Chicago's published open data, such as 2nd City Zoning, which "is an interactive map that lets you find out how your building is zoned, learn where to locate your business and explore zoning patterns throughout the city." Other apps mash up city to examine local economic trends, use school data to show parents what kind of a school district they live in, and building data to show building developments and demolitions, and violations.

Transparency and Public Shaming: Pakistan Tackles Tax Evasion

In Pakistan, where only one in 200 citizens files their income tax return, authorities published a directory of taxpayers' details for the first time. Officials explained the decision as an attempt to shame defaulters into paying up.


wednesday >

Facebook Seeks Approval as Financial Service in Ireland. Is the Developing World Next?

On April 13 the Financial Times reported that Facebook is only weeks away from being approved as a financial service in Ireland. Is this foray into e-money motivated by Facebook's desire to conquer the developing world before other corporate Internet giants do? Maybe.


The Rise and Fall of Iran's “Blogestan”

The robust community of Iranian bloggers—sometimes nicknamed “Blogestan”—has shrunk since its heyday between 2002 – 2010. “Whither Blogestan,” a recent report from the University of Pennsylvania's Iran Media Program sought to find out how and why. The researchers performed a web crawling analysis of Blogestan, survey 165 Persian blog users, and conducted 20 interviews with influential bloggers in the Persian community. They found multiple causes of the decline in blogging, including increased social media use and interference from authorities.


tuesday >

Weekly Readings: What the Govt Wants to Know

A roundup of interesting reads and stories from around the web. GO

Russia to Treat Bloggers Like Mass Media Because "the F*cking Journalists Won't Stop Writing"

The worldwide debate over who is and who isn't a journalist has raged since digital media made it much easier for citizen journalists and other “amateurs” to compete with the big guys. In the United States, journalists are entitled to certain protections under the law, such as the right to confidential sources. As such, many argue that blogging should qualify as journalism because independent writers deserve the same legal protections as corporate employees. In Russia, however, earning a place equal to mass media means additional regulations and obligations, which some say will lead to the repression of free speech.


Politics for People: Demanding Transparent and Ethical Lobbying in the EU

Today the Alliance for Lobbying Transparency and Ethics Regulation (ALTER-EU) launched a campaign called Politics for People that asks candidates for the European Parliament to pledge to stand up to secretive industry lobbyists and to advocate for transparency. The Politics for People website connects voters with information about their MEP candidates and encourages them to reach out on Facebook, Twitter or by email to ask them to sign the pledge.


monday >

Security Agencies Given Full Access to Telecom Data Even Though "All Lebanese Can Not Be Suspects"

In late March, Lebanese government ministers granted security agencies unrestricted access to telecommunications data in spite of some ministers objections that it violates privacy rights. Global Voices reports that the policy violates Lebanon's existing surveillance and privacy law, Law 140, but has gotten little coverage from the country's mainstream media.