Personal Democracy Plus Our premium content network. LEARN MORE You are not logged in. LOG IN NOW >

New York City's Latest Open Data Release

BY Miranda Neubauer | Monday, September 23 2013

NYC Open Data visualization by Chris Whong.

Property record, building permit and building footprint data for New York City are now publicly accessible to developers and the wider public as part of New York City's efforts to implement the 2012 Open Data law, several city officials announced Monday.

The city also published the Open Data compliance plan, a requirement of the law, outlining a timeline for the release of over 400 city data sets covered by the law up until December 2018.

An earlier deadline in March had required the city to make already public data available in an accessible, machine-readable format. According to the city, the Open Data portal has released over 1,000 datasets from more than 60 agencies since its launch in 2011.

Monday, the city announced that it was making available data from the Department of Finance's Automated City Register Information System (ACRIS) which allows users to search property records for Manhattan, Queens, the Bronx and Brooklyn from 1966 to the present. Also available is a superset of data that includes all permits issued by the Departments of Buildings and Health and Mental Hygiene.

The Department of Information Technology & Telecommunications (DoITT) has also partnered with Mapbox and the OpenStreetMap Community to incorporate New York City building footprints and address points into Open Street Map. DoITT will also be receiving daily OpenStreetMap updates, allowing the city to regularly improve its own data.

The Environmental Control Board has made data related to violations from 13 of the city's ticket-issuing agencies more broadly available to the public.

The Open Data compliance plan lists all data sets controlled by city agencies that the agencies have identified as falling under the Open Data law to make available to the public in an accessible, machine-readable format. Although the final deadline is December 2018, Rahul Merchant, Chief Information and Innovation Officer, and Mike Flowers, Chief Analytics and Open Platform officer, emphasized that the plan would be evolving continuously. Flowers said a key part of the process was the first-time conduction of a "data census" to get an agency-by-agency overview of available data. He said the city would be "extremely aggressive" over the remaining 100 days of the Bloomberg administration to release data, in some cases before the scheduled date. The law requires the city to publish annual updates to the plan beginning in July 2014.

The agencies covered under the plan include, among many, the Department of Education (DOE), the NYC School Construction Authority (SCA), the Department of Environmental Protection (DEP), the New York Police Department, the Department of Homeless Services (DHS), the Office of the Chief Medical Examiner (OCME), and the Mayor's Office.

By 2018, the DOE plans to release, among other datasets, daily-updated data on the status of bus routes, bus delays and daily school-wide attendance. The SCA plans to release by 2018 monthly updates on DOE building usage, as well as quarterly updates on the Classroom Connectivity program, telephone and intercom systems in schools and the Internet Protocol Surveillance System in schools. By 2014, the DEP plans to release monthly data on wastewater treatment plant performance, weekly data on water quality and monthly data on reservoir storage levels. By 2014, DHS plans to release daily data on the number of families and individuals residing in the shelter system and the number applying to the shelter system.

The NYPD plans to release annual data showing precinct-based comparisons of stop and suspect descriptions by 2014, an annual breakdown of Stop, Question and Frisk reports citywide, by borough and by precinct, by 2014, an annual report on firearm discharges by 2014, an annual breakdown of enforcement by crime and other indicators by 2016, and weekly city-wide crime statistics by borough and by precinct by 2017.

By October 2013, the OCME plans to release monthly data on the number of autopsies performed, the number of toxicology reports received and completed in sexual assault cases, and the number of DNA homicide and sexual assault cases completed. By 2014, the Mayor's Office plans to release annual data on procurements by industry, size and method, among other data.

DoITT plans to release quarterly data on its high-capacity/telecommunications services franchise agreements by 2014 and complaints made to its cable franchising center by 2017.

Not all government entities in the city are covered by the Open Data law. The New York City Housing Authority, for example, is a state-chartered public benefit corporation that is not considered a city agency under the law. Recently, Public Advocate and Democratic mayoral candidate Bill De Blasio needed to file FOIL requests to obtain data on public housing repair requests. However, Flowers said he expected more NYCHA data to be available in the future and emphasized that in general agencies have been very receptive to releasing more data. He pointed to the recent release of the PLUTO land use and geographic data at the tax lot level as an example of released data going "above and beyond what the law requires."

The NYC Developer Platform is also making available new tools for developers and data scientists, by offering APIs that allow third-party applications to integrate real-time city data. One such tool is the Geoservice API, a cloud-based geocoding service based on the Department of City Planning's Geosupport system. That system assigns coordinates, Building Identification Number and other location-based information to any entered address, intersection or place across the city. The city's Campaign Finance Board is currently using that API to offer poll site and election ballot information based on entered addresses on its mobile platform.

In addition, the city has updated the Open Data portal homepage to make it more user-friendly and to go beyond being a catalog of datasets to highlight data usage examples by businesses, students and the public.

City officials made Monday's announcements at the NYU-Poly Varick Street Incubator, which the school launched together with the city in July 2009. In their remarks, officials emphasized how the city's data releases benefit the technology community, businesses and community leaders, and encouraged public feedback to improve available data.

Micah Kotch, Director of Innovation and Entrepreneurship at NYU-Poly, emphasized that the benefits of Open Data are "not just some abstract thing." He said that teams of entrepreneurs are using the data to help develop innovative tools to solve problems for New Yorkers and people around the world, and building companies in New York. Praising the city and Mayor Bloomberg's vision and priorities, he said that the "legacy of open data...will provide the fuel, the hard currency of the innovation economy for years and years to come."

Both Merchant and Flowers not only praised Bloomberg's leadership on the issue, but also emphasized the significant role of transparency advocates. "If you folks didn't push, we wouldn't be standing here," Merchant said. "Speak up if something is missing." Evoking a Bloomberg motto that "we can do better by using data," Flowers said that "complaining is a public service." Users can suggest and nominate datasets for release through the website.

City Council Member Gale Brewer, who spearheaded the Open Data law and recently won the race for Manhattan Borough President, praised the work of the New York City Transparency Working Group, a coalition of good government organizations and technology groups that she helped bring together. She said she hoped that local Community Boards and neighborhood associations would be able to benefit from accessing the data through tools that Code for America and others create, making it "part of the discussion on the local level so we can plan," she said. Already just the list of all the agencies and datasets "is so exciting ... half of the people didn't know these datasets existed."

"Entrepreneurs, planners, scholars and advocates have waited decades for databases like PLUTO and ACRIS to be freely available to the public – and this is just the beginning.” John Kaehny, co-Chair of the NYC Transparency Working Group and Executive Director of Reinvent Albany, said in a statement. On Friday, he had said he was optimistic about New York City's Open Data progress beyond the Bloomberg administration in spite of the departure of Andrew Nicklin from DoITT to New York State.

News Briefs

RSS Feed wednesday >

Another Co-Opted Hashtag: #MustSeeIran

The Twitter hashtag #MustSeeIran was created to showcase Iran's architecture, landscapes, and would-be tourist destinations. It was then co-opted by activists to bring attention to human rights abuses and infringements. Now Twitter is home to two starkly different portraits of a country. GO

What Has the EU Ever Done For Us?: Countering Euroskepticism with Viral Videos and Monty Python

Ahead of the May 25 European Elections, the most intense campaigning may not be by the candidates or the political parties. Instead, some of the most passionate campaigns are more grassroots efforts focused on for a start stirring up the interest of the European electorate. GO

At NETmundial Brazil: Is "Multistakeholderism" Good for the Internet?

Today and tomorrow Brazil is hosting NETmundial, a global multi-stakeholder meeting on the future of Internet governance. GO

Brazilian President Signs Internet Bill of Rights Into Law at NetMundial

Earlier today Brazil's President Dilma Rousseff sanctioned Marco Civil, also called the Internet bill of rights, during the global Internet governance event, NetMundial, in Brazil.


tuesday > Reboots As a Candidate Digital Toolkit That's a Bit Too Like launched with big ambitions and star appeal, hoping to crack the code on how to get millions of people to pool their political passions through their platform. When that ambition stalled, its founder Nathan Daschle--son of the former Senator--decided to pivot to offering political candidates an easy-to-use free web platform for organizing and fundraising. Now the new is out from stealth mode, entering a field already being served by competitors like NationBuilder, Salsa Labs and And strangely enough, seems to want its early users to ask for help. GO

Armenian Legislators: You Can Be As Anonymous on the 'Net As You Like—Until You Can't

A proposed bill in Armenia would make it illegal for media outlets to include defamatory remarks by anonymous or fake sources, and require sites to remove libelous comments within 12 hours unless they identify the author.


monday >

The Good Wife Looks for the Next Snowden and Outwits the NSA

Even as the real Edward Snowden faces questions over his motives in Russia, another side of his legacy played out for the over nine million viewers of last night's The Good Wife, which concluded its season long storyline exploring NSA surveillance. In the episode titled All Tapped Out, one young NSA worker's legal concerns lead him to becoming a whistle-blower, setting off a chain of events that allows the main character, lawyer Alicia Florrick (Julianna Margulies), and her husband, Illinois Governor Peter Florrick (Chris Noth), to turn the tables on the NSA using its own methods. GO

The Expanding Reach of China's Crowdsourced Environmental Monitoring Site, Danger Maps

Last week billionaire businessman Jack Ma, founder of the e-commerce company Alibaba, appealed to his “500 million-strong army” of consumers to help monitor water quality in China. Inexpensive testing kits sold through his company can be used to measure pH, phosphates, ammonia, and heavy metal levels, and then the data can be uploaded via smartphone to the environmental monitoring site Danger Maps. Although the initiative will push the Chinese authorities' tolerance for civic engagement and activism, Ethan Zuckerman has high hopes for “monitorial citizenship” in China.


The 13 Worst Bits of Russia's Current and Maybe Future Internet Legislation

It appears that Russia is on the brink of passing still more repressive Internet regulations. A new telecommunications bill that would require popular blogs—those with 3,000 or more visits a day—to join a government registry and conform to government-mandated standards is expected to pass this week. What follows is a list of the worst bits of both proposed and existing Russian Internet law. Let us know in the comments or on Twitter if we missed anything.


Transparency and Public Shaming: Pakistan Tackles Tax Evasion

In Pakistan, where only one in 200 citizens files their income tax return, authorities published a directory of taxpayers' details for the first time. Officials explained the decision as an attempt to shame defaulters into paying up.