Boston and Cambridge Move Towards More Open Data
BY Miranda Neubauer | Thursday, April 10 2014
The Boston City Council is now considering an ordinance which would require Boston city agencies and departments to make government data available online using open standards.
Boston City Councilor At Large Michelle Wu, who introduced the legislation Wednesday, officially announced her proposal Monday, the same day Boston Mayor Martin Walsh issued an executive order establishing an open data policy under which all city departments are directed to publish appropriate data sets under established accessibility, API and format standards.
The interim Boston CIO, Justin Holmes, told techPresident Wednesday that the executive order was a continuation of policies begun several years ago that first established the Boston open data platform and an expression of new Mayor Walsh's commitment to transparency, which he emphasized in his inaugural address. "This executive order really signals a cultural shift in city government to placing a high premium on the use of data for smarter and better decisions," he said.
Holmes noted the availability of site analytics for the existing Boston platform. For the past year, the most popular data sets were closed pothole cases, crime incident reports and employee earnings reports.
The executive order empowers him to develop a set of rules to facilitate the opening of government data, Holmes said. To start off that process, the city has posted a draft policy on Google Docs that is available for public comment, where it has already gathered some feedback. The current draft establishes that a city open data manager is responsible for the publication of city data sets, with the requirement that he or she maintains a running manifest of all data published, any stipulations connected with the data sets, of all data that is in preparation for publication and data that is prohibited from publication, and make that manifest internally available to city officials.
Asked about the proposed City Council ordinance, Holmes said Walsh and the city shared Wu's commitment to and enthusiasm for open data, but was not gong to evaluate the specifics of the ordinance before it moves through the legislative process.
Speaking with techPresident Wednesday after the City Council meeting, Wu praised the city government for embracing open data with the executive order directing the CIO to begin working with departments and consulting with stakeholders to establish the open data policy.
"[With the ordinance] the council would codify the [policy] beyond the executive mandate with some additional provisions including a data coordinator within each department ... , it would mandate public participation with setting the technical standards and would require the CIO to issue an annual report back to the city and the City Council on open data and the progress in the number of data sets available," she said. The ordinance would require all agencies to make data available within one year of the ordinance's effective data. They would have to report their reasons for not disclosing data to the CIO and City Council, and the data coordinators would oversee implementation, prepare agency-specific public open data plans, a timeline and solicit public feedback online.
A similar process took place in San Francisco, she noted, where the Board of Supervisors passed an ordinance a year after an executive order from the Mayor on open data.
In her Monday press release, Wu noted that Boston's current open data portal is under no requirement to make data available beyond federal Freedom of Information requirements. She pointed out that there are only 48 items classified as data sets available, which she compared with the over 1,000 data sets that New York City's open data portal says are available since its launch in 2012. Boston's portal lists over 400 items when including items classified as charts, maps, forms and documents.
In an e-mail, Chris Whong, data solutions architect for Socrata, which manages both the Boston and New York City platforms, explained that the 48 items classified as data sets referred to fully functional "tabular data sets" accessible as a table on the Socrata platform through which users can filter or hide columns. Since "charts and filtered views" derive from a base data set, they may not count as data sets themselves, he wrote. "External data sets" refer to an entry and a link to external online resources, which could consist of many data sets, he noted. While "documents and files" are stored within Socrata and cannot be viewed or interacted with through the platform, they could still count as data sets, he suggested. With maps, he wrote that the definition might depend on whether the map is a geospatial data set, such as in KML format, or a map built based on a tabular data set. He suggested that the more accurate point of comparison for the 48 Boston tabular data sets would be the 910 tabular data sets listed on the New York City portal.
Wu also pointed to New York City's open data law as an example for requiring the solicitation of public input to develop technical standards.
Other models for the Boston ordinance were Chicago and Washington, D.C., Wu said. A group of Harvard Law students led by student Michelle Sohn conducted a bulk of the research examining the existing models, Wu said.
The ordinance is now assigned to the Council's Government Operations Committee, where the next step will be a public hearing.
While Wu said there will is a lot of enthusiasm for the ordinance among councilors, she emphasized that it would be important to balance availability and accessibility with privacy protection and public safety concerns in connection with sensitive data related to schools, public health and the police department, and explore ways to sanitize or anonymize sensitive data. Walsh's executive order also authorizes Boston's CIO to issue a Protected Data policy applicable to sensitive data not to be made public.
Wu, who stressed the need for public participation, has already reached out to Code for America brigade Code for Boston, and said there has also been a lot of interest from other parts of the local tech and start-up community. "Transparency is important, but even more important is the opportunity for collaboration and innovation from the public," she said.
Wu, who is the first Chinese-American and Asian-American woman to serve in the Boston City Council and also its currently youngest member, is chair of the Committee on Small Business, Entrepreneurship and Innovation.
Harlan Weber, brigade captain of Code for Boston, wrote in an e-mail to techPresident Tuesday that he had not yet had a chance to review the ordinance in detail yet. "At this point, I'm happy to see any motion that will empower the data organization within Boston DoIT to acquire and open more public data," he wrote, adding that he looked forward to examining the text more closely personally and with the group, and providing more formal feedback to Wu's proposal in the future.
Code for Boston has also been active in efforts to draft an open data policy for neighboring Cambridge. In the e-mail, Weber explained that the group has been working on the effort since Summer 2013, when approached by City Council member Leland Cheung, who is now a candidate for Massachusetts Lieutenant Governor. After a July committee meeting with Cheung, Cambridge and Boston city officials, academics, Code for Boston members and other members of the public, the Code for Boston community provided feedback on a Google Doc of the draft ordinance, for which Weber also solicited feedback from the broader Code for America community.
Weber noted that even as the draft was under discussion, the Cambridge GIS department published much of their geodata on GitHub, which Code for Boston drew on to develop a Cambridge voting locations web tool.
After a meeting with him, Weber and co-organizer Matt Cloyd testified at a City Council meeting last week on the ordinance as well as on a community wireless project and municipal broadband.
Weber wrote that he expected the ordinance to go through another round or two of review, and added that he hoped to incorporate some recent feedback from the Sunlight Foundation. Once the ordinance is passed, an open data citizens' group will help guide the city in determining which data sets to open and what file formats to publish them in, he wrote.