Image via CrunchBase

The more I learn about BuzzData, the less sure I am that it belongs in the Data Market club. That doesn’t mean that the site isn’t useful, or valuable, or relevant to lots of different people. Far from it. But it seems clear that BuzzData’s current focus is upon doing something very different to the other companies I’ve spoken to during this series.

Described by Nick Edouard, BuzzData’s EVP Business Development & Marketing, as a “GitHub for data,” BuzzData concerns itself with providing a place in which groups can collaborate around a set of data. This data might be a public resource that they’ve found for free on BuzzData itself, or it might be a corporate database or spreadsheet that has been uploaded to a private area of BuzzData to be worked on collaboratively. More like Huddle or Box than Infochimps or Factual, BuzzData seems primarily interested in delivering a data-centric perspective on the increasingly important task of collaboration. Whether the company can make that particular niche big enough for themselves and (if successful) the inevitable flurry of copycat competitors remains to be seen, but they’re off to a good start.

Following up on a blog post that I wrote at the start of 2012, this is the fifth in an ongoing series of podcasts with key stakeholders in the emerging category of Data Markets.

Like most of my podcasts, this one is audio-only. I am conducting a short survey this week (only one question is mandatory) to gauge interest in alternative forms of podcast, and would be grateful if you could take a moment to record your view. I shall summarise the findings on Friday.

Related articles

• Data Market Chat: Tyler Bell discusses Factual (cloudofdata.com)
• Data Market Chat: Chris Hathaway discusses AggData (cloudofdata.com)
• Data Market Chat: Hjálmar Gíslason discusses DataMarket.com (cloudofdata.com)
• Data Market Chat: Flip Kromer discusses Infochimps (cloudofdata.com)

I originally recorded a podcast with Infochimps’ Flip Kromer way back in December 2009, when most of today’s data markets were just starting out. We spoke again last week, as part of my current series of Data Market Chats, and it’s interesting to begin exploring some of the ways in which Infochimps and its peers have evolved.

Describing Infochimps variously as a “SourceForge for data” or an “Amazon for data,” Flip argues that the site’s real value lies in bringing data from different sources together in one place. This, he suggests, is part of allowing customers to “bridge the gap from data to insight.”

Despite his impatience with some of the complexities of the Semantic Web ideal, Flip willingly embraces the lightweight semantics emanating from projects such as the search engine-backed schema.org. He also recognises the value of good metadata in making data easy to use, and in introducing a degree of comparability between data sets from different sources.

Towards the end of the conversation, Flip provides his perspective on some of the other players in this space. Microsoft’s Azure Data Market, for example, is Saks Fifth Avenue to Infochimps’ Amazon; one has ‘the best’ socks, whilst the other has all the socks.

Following up on a blog post that I wrote at the start of 2012, this is the fourth in a series of podcasts with key stakeholders in the emerging category of Data Markets.

Like most of my podcasts, this one is audio-only. I am conducting a short survey this week (only one question is mandatory) to gauge interest in alternative forms of podcast, and would be grateful if you could take a moment to record your view. I shall summarise the findings on Friday.

Related articles

• Data Market Chat: Tyler Bell discusses Factual (cloudofdata.com)
• Data Market Chat: Chris Hathaway discusses AggData (cloudofdata.com)
• Data Market Chat: Hjálmar Gíslason discusses DataMarket.com (cloudofdata.com)
• Data-as-a-Service startup Infochimps swaps out CEO (gigaom.com)
• Infochimps Acquires Y Combinator Startup Data Marketplace, Expanding Brand Holdings and Online Presence (prweb.com)
• What does a single day of Flickr uploads look like as real photos? (infochimps.com)

Image via Wikipedia

Openness is undeniably cool right now, at least if you move in the slightly odd circles that I do. Openly available scientific papers are disrupting the world of scholarly publishing (which may not be all good, but that’s a post for another day). Openly available university courses are finally beginning to work out how to offer meaningful accreditation to students. Openly accessible data from government agencies around the world bulks out almost every data marketplace, and anchors many an analysis. Openly available code for cloud infrastructure or networking is challenging the hold of the tech world’s giants. Everywhere you look, ‘incumbents’ are apparently being ‘challenged’ and ‘disrupted’ by the power of open.

The truth, of course, is a little more complex and a lot more nuanced, as business models shift and evolve just like they always have. In sustainable systems, some people still need to be rewarded (often through being paid) for their effort. And in sustainable systems, paying someone can often be a pretty straightforward means of ensuring that you have a throat to choke if something breaks; big companies adopting open source often seek a proper financial relationship with someone who installs and maintains the ‘free’ software or hardware they’re depending upon.

One area of openness that I’ve been involved with for about ten years is that of open licensing for both creative works and data. And it’s come a very long way.

Here in Europe, for example, the (badly flawed) 2003 Public Sector Information Directive is under review, and there’s every likelihood that the replacement will make a number of sensible moves toward greater openness, transparency, and reusability for publicly funded data. As the EPSI Platform site notes today, Andrés Nin proposes going a step further than the European Commission is currently contemplating, by instituting a common open license across Europe;

“The creation of a single public information re-use space in Europe requires much more, it requires a common European OpenData license applicable to all data generated by European public administrations.”

I would certainly welcome a model license that European member states might be enabled to use. I’d also welcome — and support — vigorous efforts to dissuade individual member states or ministries from their usual practice of tweaking and otherwise modifying perfectly good documents in order to demonstrate how ‘special’ or ‘different’ their circumstances apparently are. When will they all realise that they are neither as special nor as different as they like to think?

But — and it’s a big but — it seems unwise, premature, and unhelpful to even begin to suggest that such a license might be mandated across Europe. It isn’t required, and attempts to develop a single document that everyone could accept would be an unhelpful distraction that would result in something so bureaucratic, so ringed in opt-outs and prevarications, as to be utterly worthless. It would also, in all likelihood, be one of those exercises in which the process very quickly subsumed the point. A prime candidate for, in the words of an old boss, being too busy to be effective.

Related articles

• Branding academic publishers ‘enemies of science’ is offensive and wrong (guardian.co.uk)
• Open Data: Europe Starts to Get It (opendotdotdot.blogspot.com)
• Open Data in Europe gets a huge boost from new EU rules (thenextweb.com)

Richard Wallis

Our conversation is also an excuse for an experiment. I have been producing audio-only podcasts here and elsewhere for a number of years, but have always tended to avoid producing video. It’s more effort, it requires more bandwidth at both ends of the conversation, and I’ve never really been convinced that it adds very much to a conversation between two people. Anecdotal evidence would also suggest that my current podcasts are consumed in environments where video would not work; washing dishes, walking dogs, and sitting on buses.

However, rather than just continue to presume that my biases are correct, I’ve decided to give video a try. Richard kindly agreed to participate, and the result is available on YouTube and embedded here.

An audio-only version is also available for download if you prefer. The introductory remarks in this version are slightly different to those on the video, as they come straight from the original conversation.

It’s perhaps unfair to draw too many conclusions from this first attempt, but a few things are immediately apparent. The whole process takes an awful lot longer. The files are larger, so processing and uploading times increase 2-3 fold. Uploading a separate audio file also takes a bit of time. Simply dumping the Skype recording into iMovie worked just fine… but I’ve (so far) not managed to find any way to balance the audio levels. Garageband lets me do this with my audio-only podcasts, but iMovie doesn’t seem to, so Richard’s side of the conversation comes across as quite a bit louder than mine.

Having done one, I’m still not convinced that the video adds anything to the conversation. But what do you think? If you’ve listened to any of my podcasts, please take a moment to complete the short survey over at SurveyMonkey. Your responses will help me to decide where to go next.

Many thanks.

Image via CrunchBase

With Iceland’s DataMarket.com, Founder Hjálmar Gíslason is on his fourth startup, and ready to expand overseas. Focused upon becoming “Google for datanumbers,” DataMarket concerns itself with collecting and providing access to quantitative data; numbers from governments, international agencies, and commercial providers around the world.

Alongside the business of collecting data and making it available for download, DataMarket has invested in providing tools with which users can visualize data (typically in the form of a graph) and even compare results from diverse sources. Hjálmar sees these tools as part of a strategy to ensure that it is

“more desirable to use data on DataMarket than at the source.”

Hjálmar also discusses his view that four characteristics of data make it profitably sellable; proprietariness, timeliness, analysis, and curation.

Have a listen to learn more about DataMarket, and to hear Hjálmar’s thoughts on an industry segment that his company has done much to shape. And check back on Tuesday for the next podcast in the series; Ian Davis of Kasabi.

Following up on a blog post that I wrote at the start of 2012, this is the third in a series of podcasts with key stakeholders in the emerging category of Data Markets. Other conversations, all of which will be published here, have been scheduled with AggData, BuzzData, Factual, Infochimps, Kasabi, and Microsoft. I am still adding conversations to the series, and intend to talk with more companies and with analysts and investors with insight to share. 

Related articles

• Nurturing the market for Data Markets (cloudofdata.com)
• Data Market Chat: the podcasts are a-coming… (cloudofdata.com)
• Data Market Chat: Tyler Bell discusses Factual (cloudofdata.com)
• Data Market Chat: Chris Hathaway discusses AggData (cloudofdata.com)

Focussed — for now — upon aggregating location data, Factual provides access by API or download to a pool of over 55 million places in the US and other territories. A key differentiator for the company is their investment in cleaning and harmonising information drawn from multiple sources. API-based services such as Crosswalk and Resolve enable developers to cope with the very different ways in which third party services like Yelp, Foursquare and Gowalla reference a single restaurant or coffee shop.

Tyler suggests, though, that location data may just be the start;

“Factual doesn’t necessarily want to be a location-only company. Really what we’re doing is we’re cutting our teeth on location now, and places… It’s just a wonderful way to learn how to refine your business and of course how to refine your technology stack… But for the immediate future, you’ll see us focus primarily on places.”

Have a listen to learn more about Factual, and to hear some of Tyler’s perspectives on the utility of good, comprehensive data. And check back on Tuesday for the next podcast in the series; Chris Hathaway of AggData.

Following up on a blog post that I wrote at the start of 2012, this is the first in a series of podcasts with key stakeholders in the emerging category of Data Markets. Future conversations, all of which will be published here, have been scheduled with AggData, BuzzData, Datamarket.com, Infochimps, Kasabi, and Microsoft. I am still adding conversations to the series, and intend to talk with more companies and with analysts and investors with insight to share. 

Related articles

• Nurturing the market for Data Markets (cloudofdata.com)
• Data Market Chat: the podcasts are a-coming… (cloudofdata.com)
• With Factual, 1 API now unlocks data for 55 million places (gigaom.com)
• New Factual Resolve API Will Help Clean Up, Complete Location Databases (techcrunch.com)
• SimpleGeo APIs Closed, But Places Data is Open (programmableweb.com)
• Quick Walk Through the World of Location & Places APIs (apievangelist.wordpress.com)
• Mobile Crunch: Urban Airship To Shutter SimpleGeo Services In March, With Factual Picking Up The Slack (techcrunch.com)

Image via Wikipedia

To follow up on my Data Markets post earlier this week, I’m now scheduling a series of podcasts in which the conversation can — and will — delve an awful lot deeper.

I’ve contacted representatives from most of the obvious data markets, some startups working in closely related areas, and several of the key analysts watching the space. So far, the response has been great. Three — Factual, DataMarket and AggData — are definitely scheduled for recording next week, and I’m waiting for firm dates from most of the others.

Through these conversations, I hope to hear more about the different ways in which the various stakeholders view this emerging market. How big is it? How valuable is it? What is their role within it? What do today’s customers look like? How do they reach and attract tomorrow’s customers? What is the mix between free and commercial data? Where are the sustainable business opportunities? What are the barriers to adoption? What is the mix between providing data, providing tools, and providing insight?

If there are other questions that you would like answered, or interesting companies you’re worried I’ve missed, either let me know in the comments or get in touch directly.

The first podcast will be published next week, and I currently aim to release one a week after that until they’re all out. I’m looking forward to these conversations, and hope that you’ll find them useful.

Related articles

• Nurturing the market for Data Markets (cloudofdata.com)

Image of Stephen Wolfram via Wikipedia

British-born computer scientist Stephen Wolfram sees ongoing efforts to extend the Internet’s top-level domains (TLDs) beyond the familiar .com, .org, .uk etc as an opportunity to raise the profile of machine-readable data. In a blog post published yesterday, he argues that a new .data domain would increase “exposure of data on the internet—and [provide] added impetus for organizations to expose data in a way that can efficiently be found and accessed.” Whilst wholly in favour of Wolfram’s stated aim, I can’t help feeling that his suggested solution is at best unnecessary and at worst a worrying segregration of data from the ‘proper’ web that everyone else will continue to exploit.

Back in June of last year, the body responsible for coordinating the global domain name system approved a plan to permit new top-level domains (the letters after the final dot in an internet address — the .com in cloudofdata.com, the .uk in bbc.co.uk, the .edu in harvard.edu). Until recently, these top-level domains have been tightly controlled, with a small set of generic domains (.edu, .gov, .mil, .org, etc), a larger set of country domains (.uk, .fi, .nz, etc) and one or two others such as .eu. From tomorrow, anyone with $185,000 will be able to submit a proposal to create and manage a new top level domain, and it’s possible that there could eventually be thousands of them. Wolfram is keen to ensure that data doesn’t miss out on the ‘opportunity.’

As Wolfram himself recognises, there is already an awful lot of machine-readable data on the web. Some of it sits embedded within the web pages that humans read, with specially formatted code waiting to be triggered by the calendars, the address books, or the browser plugins of site visitors. Some of it is packaged up in data files, offered for download. And some of it waits inside a database, ready to be delivered in response to an API call or a query typed into a web form.

There is a growing enthusiasm for exposing this data for reuse. Government transparency agendas have driven public sector data sites like data.gov.uk and data.gov. Similarly, efforts such as data.open.ac.uk and data.southampton.ac.uk see universities beginning to consciously collect data sets together and offer them up for reuse. Similar efforts in the commercial world are less easy to point to, but that reticence has nothing whatsoever to do with the lack of a ford.data, boeing.data, ge.data or astrazeneca.data domain!

In some ways, the convention for gathering significant chunks of data on a data.xxx.yyy site echoes Wolfram’s intention, but with a number of advantages. Data without context is far less valuable than data with context. Much of that context may be inferred from the domain in which the data lives, with data delivered from a .gov or .edu (or .gov.uk or .ac.uk) site perhaps interpreted differently to data hosted on .com, .biz, or .xxx. Southampton University, the Open University, and the US Federal Government are able to gather data up and make it available for download via their existing data. sites if they choose. This offers human visitors to their sites a degree of convenience, whilst retaining the power and brand attributes of their existing domain. Gov.data, gov.uk.data, open.ac.uk.data, southampton.ac.uk.data, though? All are messy, in ways that Wolfram’s own wolfram.data would admittedly not be, and all are simply additional registrations that the institutions would have to pay for in order to stop someone else grabbing the domain.

At the end of the day, the machines don’t actually care. The existing data.open.ac.uk-type sites are human conveniences, not machine enablers. The computers, and the software they run, are quite capable of crawling the public web and finding accessible data wherever it lies on a site. There are plenty of reasons to continue embedding little snippets of data inside human readable web pages, regardless of whether you have a data.wolfram.com or a wolfram.data site. Content negotiation is becoming increasingly capable, such that there really is no need for what Wolfram calls a ‘parallel construct to the ordinary web’ at all. A human being arriving at a web site sees human readable content, whilst various software tools would automatically be presented with very different data or functions, optimised to their capabilities and requirements.

By all means, let us show the curious some of the existing techniques that work in making data more easily accessible. By all means, let us identify the gaps, the issues, the problems (none of which a new TLD even begins to address). Yes, let us definitely and unambiguously set about “highlighting the exposure of data on the internet—and providing added impetus for organizations to expose data in a way that can efficiently be found and accessed.”

But please, let us not be distracted by the false hope that adding yet another TLD to the babel that ICANN is about to unleash can do anything more than consign data to some online ghetto, wallowing unwanted, unloved and unused as companies and their customers lavish love, attention, and clicks upon the .com domain over on the ‘proper’ web.

Thanks to Raphaël Troncy, whose tweet first drew the story to my attention.

Related articles

• Is It Time For Computers To Have Their Own .Data Domains? (techcrunch.com)
• ICANN Pushes Ahead With January 12 Launch For New Top-Level Domains (wired.com)
• The biggest change in DNS since Dot-Com (wired.com)

In many ways, the core concept of the data marketplace is nothing new. Companies like Bloomberg, Nielsen and Experian have built (extremely) profitable businesses by aggregating data, quality checking it, and selling it on. Often their customers could have gone directly to the source(s) and paid far less, but they don’t. The convenience and quality assurance of dealing with a single — reputable — source is perceived to have value. A brand like Bloomberg’s is associated with trustworthiness and authority, and the brand of the marketplace is far more prominent than the data sets upon which it is built.

Similar sites have also served the needs of those seeking data for free, with IBM’s ManyEyes project, Freebase (acquired by Google), Hans Rosling’s Gapminder or The Guardian‘s Data Store amongst those typically mentioned. Current government enthusiasm for ‘transparency’ has fed all of these sites with data, and led to creation of large government-specific data repositories such as data.gov.uk.

The commercial services like Bloomberg have tended to focus upon specific domains (finance, in Bloomberg’s case) or types of data. They have also tended to be eye-wateringly expensive; aimed squarely at the small market segment for whom the data are mission-critical and the fees are affordable. The free services like Gapminder also tend to focus (global development statistics in this case). Other, perhaps, than experiments like ManyEyes, both the free and the commercial sites tended to aim for a degree of comprehensiveness and authority. They wanted to become the place to turn for their type of data.

But for the new generation of data markets, the picture becomes far less clear. They tend to be catholic in their data acquisition policies, they typically don’t even attempt comprehensiveness, they mix free (almost all of them hold identical large swathes of government data from the US, the UK, and elsewhere) with commercial data, and they continue to feel their way toward business models that might prove sustainable for the long haul. Perhaps more seriously, they appear almost schizophrenic with respect to brand projection, attempting to push both their own brand and those of the data sets they host in ways that can confuse far more often than they enlighten.

In attempting to differentiate themselves, today’s data markets are seeking to add features and functionality in order to be seen as far more than simply places to buy third-party data. They want to become recognised for quality assurance, for data enrichment, or for tools and capabilities that make working with the data easier or more powerful. They want to become sticky, and they want to be seen as different from their competitors. The trick, though, is to explain those features and those differences in ways that make sense to potential customers. Those customers will ultimately pay for functionality and utility, not for gimmicks or under-the-hood technological distinctions that have no real impact upon getting on with the job in hand. Are today’s data markets describing their features in ways that help prospective customers to understand why they should be chosen over the alternatives? Not really. At least, not yet.

Also, as RedMonk’s Stephen O’Grady touched upon amongst a set of related issues, we’ve really not begun to see much evidence of price competition. There are too few suppliers, each with their fiercely loyal bands of tame users (‘customers’), and too few people prepared to shop around for the best deal.

The new data markets are still young. Understandably, they are still feeling their way in order to understand what the market wants, how much it is prepared to pay for what it wants, how large the market might be, and what their individual niche within that broader market might look like. Earlier models, based upon almost monopolistic domination of specific verticals and polarised pricing, offer some lessons but are ultimately unsatisfactory blueprints for this more competitive, open, and complex environment. Beyond specific domains like finance (which may be ripe for disruption), the data markets must struggle to convince prospective customers that they have something of value to offer. Those customers may already have their own processes for obtaining data. They may generate the data themselves, or expect — as so many do — to be able to access what they need for free. They are perhaps suspicious of data produced by third parties who are, in other contexts, their competitors, and they are almost certainly unwilling to allow ‘the competition’ to benefit from their own data. They invariably do not understand the costs associated with gathering and quality-assuring data, or the challenge of preparing different data sets in order that they may meaningfully be combined. And into this, the fledgling data markets must insert themselves, market themselves, and sell themselves. They must change behaviours, they must challenge presumptions, they must alter working practices, and they must persuade their new customers that all of this pain is worth paying for. A tall order, indeed, but necessary if any of them are to realise their potential.

The European Commission, at least, begins to comprehend the scale of the challenge. A set of projects are currently being finalised, and this year will see European SMEs given the funding to boot-strap a number of new data sources. With Commission funding, it is hoped, the chosen projects will be able to explore models by which data can be created, curated, shared and re-used in a manner that is cost-effective and ultimately sustainable. The funding should enable these projects to reach viable scale, and give participants the freedom to explore alternative commercial models. The projects will be announced shortly, but only time will tell if the funding and the incentives are sufficient to break through the barriers that prevented any of these markets from forming by themselves.

But outside the rather artificial bubble created by European public funding, there is a lot of work to do. Investors are intrigued by — but still wary of — the opportunity. Infochimps is spending its way through over $1.5 million of investment, Factual has almost $30 million, and companies like Talis and Microsoft are making not-insignificant investments in their own efforts. We’re all still experimenting, but with the real market for these services currently falling far short of the money at stake, it mustn’t be long before investors start asking harder questions. Back in 2010, Pete Soderling and Pete Forde described data as a $100 billion market. The data markets may be after a significant chunk of that but, today, they’re not even close.

The ways that data markets are attempting to differentiate themselves, and the work being done to understand the market opportunity here, will have to wait for subsequent posts.

Disclosures: I am a former employee of and current shareholder in Kasabi’s parent company, Talis. The European Commission is, from time to time, a client.

Related articles

• With Factual, 1 API now unlocks data for 55 million places (gigaom.com)
• ‘Roswell’: Another key component of Microsoft’s cloud strategy (zdnet.com)
• Infochimps Acquires Y Combinator Startup Data Marketplace, Expanding Brand Holdings and Online Presence (prweb.com)

Image via Wikipedia

For real-world applications of Linked Data and the Semantic Web, the long-running Semantic Technology Conference is hard to beat. For getting a real handle on the Cloud Computing landscape, GigaOM‘s Structure Conference is also a leading light.

Working across both areas as I do, these events tend to figure prominently in my calendar for the year*. Last year, both took place in San Francisco during the same week. I tried to attend both, and therefore succeeded in spending most of my week in cabs, shuttling between meetings at the two venues. I saw very few sessions that I wasn’t personally involved in, and the experience wasn’t a huge success.

This year the conference organisers have taken pity on me, and moved their events to opposite ends of June. The Semantic Technology Conference is up first, back at the Hilton Union Square from 5-9 June. Structure follows, returning to the Mission Bay Conference Centre on 22 and 23 June.

I’ll be attending both, and probably doing various official things during each event. At the moment, the only thing we’ve definitely nailed down is a special live appearance by The Semantic Link crew on the evening of 5 June; we’ll be taking a look at the highlights expected for the conference, offering some tips for those new to the event and its multitude of parallel sessions, and generally bringing our usual podcast chatter to the stage.

So if you’re in town around the time of either event, do get in touch; it promises to be an interesting month.

* Disclosure: WebMediaBrands pay me to host the monthly Semantic Link podcast, and to write a monthly column on SemanticWeb.com. GigaOM pay me to curate the Infrastructure/ Cloud Computing channel on their Pro site. I attended and participated in both of these events before that was the case, and still would today without the contractual relationship.

Related articles

• GigaOM Structure 2011 (datacenterknowledge.com)
• Semantic Technology Conference (fakeiitian.com)