DataMarket blog

Data, visualization and startup life

Why Open Data is all about Apps, and why it shouldn’t be!

with 8 comments

Open data initiatives rock. In fact, without the trend of government and international organizations releasing their data under open licenses, DataMarket.com wouldn’t be so incredibly interesting. So obviously we love them!

Yet, I have something of a grudge against the emphasis on very specialized apps in Open Data initiatives. “Apps for this”, “Apps for that”, competitions, cute little prices, etc., etc.

Now don’t get me wrong, many of these apps are great, but they only release a tiny fraction of the value in all the data that has been opened up. That’s certainly true of each single app, but it’s also true of them in aggregate. Here’s why.

Most, if not all of the data that has been opened up, has been published in a format that is relatively accessible to developers and other data savvy people, but not so much for consumption by mere mortals. Therefore, in order for a successful app to emerge, three things have to come together as depicted in this Venn diagram:

  • Data has to be available
  • There must be obvious user demand
  • And as developers are the “keymakers” to all this data, there must be some developer incentive, be that money, coolness, recognition by peers or all of the above

Looking at the examples of successful open data apps out there, this pattern becomes quite obvious. Take the plethora of city data that has been opened up over the last 3 years or so. An overwhelming majority of the successful apps created on top of this data are transportation apps. All three elements are there. The data, the obvious need by millions of people and the developer incentive to: scratch their own itch, create awesomeness and make money (roughly in order of priority). And these apps are cool, I use some of them almost every day!

However, because of the three requirements mentioned earlier, there is such a great portion of the data that has been opened that is still just lying around unused. Again, Venn:

I’ll use another example of city data to explain: Sewage data. The data is there, but the demand may not be obvious, and there’s nothing sexy about making this data more accessible. I mean: “Who loves sewage information?”

Tell you what. I’m sure that if this data could be made better available to:

  • …construction workers to prevent pipe cuts
  • …environmentalists and policy makers to improve the regulatory environment
  • …advertisers to calculate the percentage of the half-game audience that missed their ad when taking a leak

…the overall social and economic benefits of better access to sewage data alone could be quite dramatic. And that’s just one example.

So, what I’m getting at is this: When thinking about Open Data initiatives, think beyond the apps. Think not only about the high publicity use cases that are a worth a few dollars to millions of users. Think also about the less sexy cases that can help a few people save us millions of dollars in aggregate, generate new insights and improve decision making on various levels.

Think about how you can encourage data portals making the bulk of your data accessible to mere mortals, not only to developers. Think how you can get existing software vendors to integrate your data, and how you can make business users and other decision makers aware that this data indeed exists.

There could be more to Open Data than a bunch of cool consumer apps.

Written by Hjalmar Gislason

May 22, 2012 at 5:40 am

Posted in Uncategorized

8 Responses

Subscribe to comments with RSS.

  1. [...] Why Open Data is all about Apps, and why it shouldn’t! « DataMarket blog – from DataMarket blog http://blog.datamarket.com [...]

  2. [...] Programming News: Why Open Data is all about apps, and why it shouldn’t be!    Open data initiatives rock. In fact, without the trend of government and international organizations releasing their data under open licenses, DataMarket.com wouldn’t be so incredibly interesting. So obviously we love them!     Read full story => DataMarket [...]

  3. You’re making a really good point – the apps are the shiny thing and it’s shiny which invariable attracts people to a thing.

    However, I think that in this particular chicken and egg situation you need to have the shiny in order to make the potential of open data real to those who might need to make the decisions.

    At Hull City Council we’re really keen to see our publication of open data become a product of having systems that are fit for purpose and which help us to shape our internal decision making just as much as giving local developers the chance to get their hands dirty.

    Had one of my colleagues not used our published spending data and some open sourced code to build http://www.upnorthauditor.com then it’s unlikely that this approach would have been supported. But because it took something managers usually saw in spreadsheets and turned it into something more visual they began to get a sense of what’s possible.

    Those of us in the public sector choir who get why opening data is important and appreciate the wide variety of ways it can be used need ways of sharing that with others. Apps can do that.

    In Hull we’re building relationships so that when the obvious apps emerge we can still foster conversations, interest and opportunity to reveal the less obvious secrets hidden within our massive data.

    Gosh, this turned into an epic comment!

    bmwelby

    May 22, 2012 at 5:38 pm

  4. great observations.

    IMO, an important focus that doesn’t get enough attention is making data “use-able.” raw data is not use-able. we need to take the methodologies developed making user interfaces use-able and apply that to the data itself.

    Data is not truely Open until it is Use-able.

  5. The CIO of Palo Alto California just gave a talk about how they’re embracing technology, specifically apps: http://www.mobilecastmedia.com/2012/05/city-of-palo-alto-embraces-apps/

    John Houghton

    May 24, 2012 at 3:58 am

  6. Some really good points in this blog post. I think Mike makes a good comment though — access is not equal to usability. The road from input to insights is mired with non-trivial learning curves, data normalization and transformation concerns, a variety of specialized tool sets (hadoop, hive, flume, tableau, R, etc.) putting the onus on integration work you must do, etc.

    Apps are sexy and the thing you can see (versus the often more ominous, murky world of raw data) However, the overall success of the entire “big data” market (which, to link to Gartner’s age-old framework, may mean velocity of fast data, volume of massive data, or variety of data sources and formats) depends on *apps* to grow in power and adoption, the plethora of *data* sources big, small, public, and private to be at your fingertips, and *developer* knowledge, buy-in, open-source + proprietary contributions, and budgets -> to all be applied, nurtured, and expanded in unison.

    Tim Gasper

    May 24, 2012 at 4:20 pm

  7. [...] 'UA-22979035-1', data_ga_social: true }; Hjalmar Gislason commented earlier this year that open data has been all about apps. To capturet should be about much more than consumer-facing tools. “Think also about the less [...]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 55 other followers

%d bloggers like this: