NYPD Among First To Release Detailed Accessible Local Collision Data (Updated)
BY Miranda Neubauer | Thursday, May 8 2014
The New York Police Department has published long-sought motor vehicle collision data in a machine-readable format in connection with the launch of BigApps 2014, the city's annual application development competition. This year's competition will focus on Mayor Bill de Blasio's Vision Zero initiative to eliminate traffic fatalities, setting an example for other cities, according to open data advocates.
The data will be updated Monday to Friday on a three-day-lag, according to Lara Torvi, spokesperson for New York City's Department of Information Technology & Telecommunications, which helps oversee the open data portal.
"This is amazing, and more than what we were expecting — the geocoding and integration with the data portal are unexpected and *extremely* pleasant surprises," betaNYC, New York City's Code for America brigade, said in a statement from the group's executive director Noel Hidalgo and John Krauss. Krauss is a developer who had scraped intersection collision data from NYPD PDFs to create a Crashmapper platform due to the limited information available previously. That data was the basis for a recent New York Times graphic on traffic accidents.
Hidalgo and Krauss added, "This dataset allows us to stop second guessing NYPD’s monthly reports, and focus on improving this dataset’s capability. Now, we must push for better collision reporting at the scene."
In the statement, they emphasized that betaNYC has been advocating for the release of such data since 2011. The group renewed its push for more traffic data in connection with the announcement of de Blasio's Vision Zero plan. "Today, we commend the NYPD for placing detailed and machine readable crash data online," they write in the statement. "Additionally, we are ecstatic that Mayor de Blasio shares our vision. We are excited to see the Mayor challenge NYC’s BigApps participants to build tools to achieve Vision Zero."
They also pointed out several areas with room for improvement. While they highlighted the fact that the data released goes back to July 2012, they also noted that that seems like an "arbitrary cut-off point," since data is archived back to August 2011. They also pointed out that the dataset is missing information on the total number of vehicles involved in an incident and which contributing factor applies to which vehicle, and urged the NYPD to provide more detailed location data.
"It is awesome that the NYPD is geocoding these records, but the root of the data is still the intersection. This causes the loss of incident clarity," they wrote.
In an earlier e-mail, Krauss noted that previously the data was only available in non-machine-readable PDFs and Excel formats, only had aggregate information by month and only had information on street names without longitude or latitude data.
He also added that he would like to see moving violations data, in order to be able to "correlate collisions to NYPD enforcement action."
NYC Moves Ahead
Cities that already appear to release some traffic accident data on a daily basis include Seattle, which offers basic location data on police responses to 911 traffic accident incident reports refreshed on a four hour interval, and Denver, which has daily updated traffic accident location data. In Chicago, civic hackers have built the Chicago Crash browser with location and injury information, based on annual data from the Illinois Department of Transportation. In Washington D.C., the Twitter feed StruckinDC retweets and counts accident location reports tweeted by the official DC Fire and EMS feed. But while cities such as Boston, San Francisco, Los Angeles, Oakland, Philadelphia, Chicago, Washington D.C. may be ahead of New York City in providing crime data according to the Open Knowledge Foundation and Reinvent Albany, their open data portals do not appear to host comparable machine-readable data sets devoted explicitly to detailed traffic collision data.
"I think what NYC has done is awesome and I'm not aware of any other city that has made this kind of a commitment to sharing vehicle collision data," Mark Headd, former Philadelphia Chief Data Officer and now developer evangelist at Accela, wrote in an e-mail to techPresident.
As that city's Chief Data Officer, he said, working for the release of traffic data was difficult because different city and state departments overlapped in their responsibilities for collecting the data and there were different reporting standards. "Adding to the complexity here, depending on where the crash occurs there may be some debate as to who captures that information and is required to report it. Big cities like NYC and Philly have lots of colleges & universities - some with their own sworn police forces," he wrote.
Headd added, "Also, consider that if an accident occurs as a motorist travels from a city street onto a highway - whose responsibility is it to collect the data? The city police - who patrol the city streets - or the State Police - who patrol the highways and interstate? In what data set is the incident captured?" He noted, however, that the release of crash data is still on the agenda of Philadelphia's Open Data Pipeline.
In the meantime, he pointed out that a local journalists and a civic hacker had obtained a dataset of crash locations from the Pennsylvania Department of Transportation for Philadelphia County from 2008 to 2012.
"Here in Philadelphia, the issue of pedestrian crashes has largely failed to earn the status of a political issue, partly because there is no broad coalition here analogous to New York’s well-organized livable streets movement, but partly because there has not been usable data on the problem — until now," journalist Jonathan Geeting ">recently wrote in a piece for Next City, citing de Blasio's Vision Zero initiative.
"NYC has set a good example for other cities to follow, and I hope other cities follow their lead," Headd wrote.
"This is a great step towards realizing Mayor de Blasio's Vision Zero initiative," said Noah Budnick, deputy director of Transportation Alternatives. "The first step in...evaluating traffic safety is knowing where are the most dangerous corridors and what are the most frequent contributing factors that lead to people being killed and injured on the street."
Releasing the data "is a great way to engage New Yorkers around making our streets safer and will also hopefully allow a conversation about how to target resources and lifesaving measures in the most efficient and cost-effective ways." When the data was previously released only by intersection, it would only show that five crashes occurred at one location, but would not offer details of each crash, he explains, as he praised the current momentum on the issue in New York City. "You have open data, you have the tech community, and you have Vision Zero, all three very progressive ideas coming out of New York City right, so it's exciting to see how they're going to combine ...."
Budnick noted that Transportation Alternatives had first undertaken an effort to compile and publicly map crash data ten years ago, creating a Crashstat web page using data from the New York State Department of Motor Vehicles data that was updated every 18 months. Since then, it's been a process "of getting more timely, cleaner and more machine readable data," he said.
According to the NYCEDC press release, de Blasio's call for technological solutions for traffic safety is one of the over 30 BigApps challenges. In a video, he says New York City's homepage will feature the challenge's winning application and that he will thank the developers in person at City Hall.
The other week, the city's Department of Transportation had called on New Yorkers to provide feedback on dangerous intersections through an interactive map on OpenPlans' Shareabouts platform.
Developers that are active in betaNYC have wasted no time in visualizing the new data. Chris Whong, data solutions architect at Socrata and co-captain of BetaNYC, created a visualization of collisions in his Brooklyn neighborhood. Andrew Hill, a data scientist at CartoDB, created maps of people injured or killed across New York City, contributing factors, collision hotspots and injuries by numbers.
And more transparency could be underway soon from the City Council as well. Wednesday afternoon, the Council's Rules Committee held a hearing on legislation that would mandate that legislative and discretionary funding data be available to the public in a machine readable format, and call on the City Council Speaker to develop a public technology plan for public access to City Council materials and meetings.
During the hearing, Council member Ben Kallos, the chair of the Government Operations Committee who has put an emphasis on open government, announced that the Sunlight Foundation would be partnering with the City Council to help make its data more accessible.
Earlier this year, new Police Commissioner Bill Bratton had promised that the department would put a new emphasis on transparency. Tuesday, de Blasio had announced the appointment of Anne Roest as commissioner of the Department of Information Technology and Telecommunications.
The Daily News reported Wednesday that pedestrian fatalities had decreased by one third in in first four months of 2014.