(I also posted this over on the OpenPlans blog)
A few weeks back, I sent out a solicitation to a bunch of city clerks and other people connected to City Council bodies around the country that use Granicus's legislative management software, Legistar. I was requesting signatures on a publically available letter as a statement of the importance of opening the legislation data that Granicus collects through Legistar. This type of data has been a persistent interest of mine ever since I helped start Councilmatic a couple years ago.
The Method
To find the cities that use the software I searched Google for all sites on the domain legistar.com (i.e., with the search query "site:legistar.com"). Then I wrote a little scraper to step through the pages of that search and collect the subdomains. I took the results and started collecting contact information for the City Council contacts in those cities (see those results here — it's a far-from-complete list). I then contacted the people in that list with a little bit of background about myself, my projects, and the letter's purpose:
The letter is meant to let Granicus know that their clients think that it is important to have their data open and available to the public, specifically through an API. The letter is directed at Granicus because, as the letter states, they are in a unique position to open this data for a number of bodies all over the country at one time. The developers at Granicus are in favor of this type of action, but we believe that hearing from their clients that this is important would help them to prioritize it further.
...Many opportunities for business enterprise, journalistic storytelling, public advocacy, and citizen experimentation are lost when there is no access to quality data sources.
For demonstration of the latter point, see amazing work done by the likes of Open City folks in Chicago, or AxisPhilly in Philadelphia, or work coming out of the Code for America fellowship. There are many more examples of applications built with open data (here's a list of seven more apps covered by Mashable), but the data has to be out there in a way for developers and tinkerers to work with first.
An additional, unstated purpose was just to raise awareness about the value of opening this data, particularly with so many different parties interested in local legislative data recently.
The Response
I received responses from 5 of the people on the list, two from cities that agreed to sign. Both of the cities with favorable responses were from people that I had some prior connection to, at least through some common acquaintance, underscoring the importance of personal relationships and trust when making changes. I think that it is reasonable to expect a low return rate for any cold-call communication. The person on the receiving end is even less likely to response, I think, when they don't understand the purpose of the solicitation. We're still far from a point where an open data vocabulary is common in government. For example, one of the contacts that got in touch with had this response:
Please note that any documents pertaining to the [city represented] that would be available through Granicus/Legistar are already available to any member of the public through our website. I am fairly confident that is true of any city that utilizes their services. Have a good day and a wonderful new year.
The benefits of making data open and available in raw formats is something that I have found difficult to communicate outside of one-on-one conversations. It is definitely a big step to make data available in a human readable format, but computers aren't good at getting information in the same ways that humans are. Enabling people to build new things, tell new stories, and otherwise make the data useful to more people in more ways requires access to the data in a more computationally malleable form.
What's most frustrating is that, often, data is already sitting in a repository somewhere ripe for developers to use. This is evidenced by the fact that many documents, such as every Legistar legislation record, are generated from raw data. That data is simply inaccessible in its raw form. Access to the data is mediated by the format that the document is rendered into. Scraping is an option in this case, but it is an overly complicated, inherently fragile solution, and there are better solutions to be had.
Data openness initiatives like these have a higher chance of success when there are local supporters who understand the issue and can coach city contacts through what it all means. This is the approach that the OpenCity group is taking in Chicago, with some success. If I did this over, I would start with cities that have a local Code for America brigade to help with the outreach.
