Image via Wikipedia

Openness is undeniably cool right now, at least if you move in the slightly odd circles that I do. Openly available scientific papers are disrupting the world of scholarly publishing (which may not be all good, but that’s a post for another day). Openly available university courses are finally beginning to work out how to offer meaningful accreditation to students. Openly accessible data from government agencies around the world bulks out almost every data marketplace, and anchors many an analysis. Openly available code for cloud infrastructure or networking is challenging the hold of the tech world’s giants. Everywhere you look, ‘incumbents’ are apparently being ‘challenged’ and ‘disrupted’ by the power of open.

The truth, of course, is a little more complex and a lot more nuanced, as business models shift and evolve just like they always have. In sustainable systems, some people still need to be rewarded (often through being paid) for their effort. And in sustainable systems, paying someone can often be a pretty straightforward means of ensuring that you have a throat to choke if something breaks; big companies adopting open source often seek a proper financial relationship with someone who installs and maintains the ‘free’ software or hardware they’re depending upon.

One area of openness that I’ve been involved with for about ten years is that of open licensing for both creative works and data. And it’s come a very long way.

Here in Europe, for example, the (badly flawed) 2003 Public Sector Information Directive is under review, and there’s every likelihood that the replacement will make a number of sensible moves toward greater openness, transparency, and reusability for publicly funded data. As the EPSI Platform site notes today, Andrés Nin proposes going a step further than the European Commission is currently contemplating, by instituting a common open license across Europe;

“The creation of a single public information re-use space in Europe requires much more, it requires a common European OpenData license applicable to all data generated by European public administrations.”

I would certainly welcome a model license that European member states might be enabled to use. I’d also welcome — and support — vigorous efforts to dissuade individual member states or ministries from their usual practice of tweaking and otherwise modifying perfectly good documents in order to demonstrate how ‘special’ or ‘different’ their circumstances apparently are. When will they all realise that they are neither as special nor as different as they like to think?

But — and it’s a big but — it seems unwise, premature, and unhelpful to even begin to suggest that such a license might be mandated across Europe. It isn’t required, and attempts to develop a single document that everyone could accept would be an unhelpful distraction that would result in something so bureaucratic, so ringed in opt-outs and prevarications, as to be utterly worthless. It would also, in all likelihood, be one of those exercises in which the process very quickly subsumed the point. A prime candidate for, in the words of an old boss, being too busy to be effective.

Related articles

• Branding academic publishers ‘enemies of science’ is offensive and wrong (guardian.co.uk)
• Open Data: Europe Starts to Get It (opendotdotdot.blogspot.com)
• Open Data in Europe gets a huge boost from new EU rules (thenextweb.com)

For various reasons, I was unable to make it to Virginia for the event, but a scan through the presentations from Leigh, Tom Heath (another former colleague), Jordan Hatcher (with whom I worked on earlier iterations of the Open Data Commons license), and Creative Commons‘ Kaitlin Thaney, it looks like they did a great job covering the bases on this critically important aspect of the evolving Data Web.

If we are to encourage the sorts of break-out use of data that Tim Berners-Lee and Tim O’Reilly were discussing at last week’s Web 2.0 Summit, we need to move past the current laissez-faire approaches adopted by too many and ensure that there are licensing regimes in place to enable, facilitate and encourage widespread re-use.

As Leigh’s work demonstrates, we have a long way to go. Only around a third of the current Linked Data projects are explicitly licensing their content, and several of those may be mis-licensing by applying copyright-based licenses such as those from Creative Commons to data not covered by copyright legislation.

With the current state of data licensing, and the reliance of CC0 and Open Data Commons’ Public Domain Dedication and License upon the public domain as a means of overcoming territorial differences, will we find enterprise use of Linked Data increasingly relying upon the more cumbersome tool of contract law… to the detriment of a free and flexible exchange of ideas?

‘Linked Open Data Rights Survey‘ image by Leigh Dodds, shared on Flickr under a Creative Commons license.

Related articles by Zemanta

• ISWC linked data and the law (opencontentlawyer.com)
• Iswc2009 I. (ivan-herman.name)
• CC0 endorsed in Nature opinion piece (sciencecommons.org)
• Linked Data: Interpretants and Interpretation (phaneron.rickmurphy.org)

My latest podcast is with John Wilbanks, the VP at Creative Commons with responsibility for their Science Commons project.

John has a varied background that includes founding a bio-informatics startup, Harvard’s Berkman Center, the World Wide Web Consortium and the US Congress.

In his current role at Science Commons, he is working to ensure that the outputs of publicly funded science become more available; both for other scientists to use, and for the wider public. The successes of the Open Access movement have led to greater visibility for scientific papers, but the data upon which those papers depend still tends to be difficult to locate.

We discuss initiatives at Science Commons and elsewhere, and consider some of the barriers to a change in approach.

Production of this podcast was supported by Talis, and show notes are available on their Xiphos blog.

Related articles by Zemanta

• MIT opens access to its research articles (cbc.ca)
• Wilbanks’ talk on ‘Knowledge Interoperability’ from CSE now online (sciencecommons.org)
• Open Science, Closed Source (opendotdotdot.blogspot.com)
• Sage – Open Access Data from Merck (joi.ito.com)

A short post, sparked by two related items that arrived by Twitter and email almost simultaneously.

Via Twitter, TechnologyReview reports that it’s getting easier to visualise massive data sets without the traditional supercomputer.

Via email, the UK’s Joint Information Systems Committee (JISC) announces its contribution of £200,000 to a joint funding call with the United States’ National Science Foundation (NSF) and National Endowment for the Humanities (NEH), and Canada’s Social Sciences and Humanities Research Council (SSHRC). The scope of the call? The challenge of working with Big Data. I’m very keen to see the bids to that particular call… and to see the innovation that answering its challenges will (hopefully!) spark.

Add to that my post last month about Amazon’s Public Data sets, and  the signs are getting ever-stronger that there are real opportunities here.

It’s getting easier to manage massive data sets. Those data sets are increasingly addressable over the Web. There’s growing interest in working across and between data sets, and there are even licenses to ensure that researchers can do what they need to.

Interesting times, indeed.

It began, as so many things do these days, with an idle tweet.

On 21 November, Amazon Web Services‘ Deepak Singh pointed to a new page describing the company’s ‘Public Data Sets on Amazon Web Services.’

Lidija Davis covered the news for ReadWriteWeb two days later and on 4 December Amazon issued its formal press release, prompting a flurry of coverage from Mike Arrington at TechCrunch, Larry Dignan at ZDNet, Krishnan Subramanian at CloudAve, and many others.

Alongside broader discussion of this move, members of the W3C-backed Linking Open Data project delved into the synergies via their public mailing list and Linking Open Data enthusiast Kingsley Idehen‘s company issued a Press Release suggesting ways in which their products might fit within this shifting data landscape.

So what have Amazon done, what does it mean, and how does it ‘bring the Cloud of Data closer’ as the title of this post suggests?

Amazon’s web page describes their offer quite succinctly;

“Public Data Sets on AWS provides a centralized repository of public data sets that can be seamlessly integrated into AWS cloud-based applications. AWS is hosting the public data sets at no charge for the community, and like all AWS services, users pay only for the compute and storage they use for their own applications.”

As Krishnan noted in his post,

“By doing this, Amazon is helping research community save money on storage and bandwidth costs associated with assessing these public data from any EC2 instances they use in their research. When the data in question is in hundreds of terabytes or petabytes, we are talking about huge cost savings here.”

In addition, OpenLink’s press release gives an indication of the efficient manner in which services and data already hosted by Amazon can be plugged together as needed;

“As a vital contribution to the momentum behind the burgeoning Web of Linked Data, [OpenLink's product] Virtuoso provides a simple deployment mechanism for highly integrated knowledge bases emerging from the Linking Open Data community. For example, it is now possible to deploy personal or service-specific renditions of DBpedia within 1.5 hours, compared to an 8 – 22 hour effort when performed from scratch.”
(my links)

By offering free hosting for public data, then, Amazon are doing the wider community a huge service. Much of the data there today is reasonably readily available from other sources, so the biggest immediate benefits are those of speed and cost outlined above by Krishnan and OpenLink. For existing or potential users of Amazon’s Web Services to power their applications, this is yet another reason to consider Amazon.

Harvard Medical School‘s Dr. Peter Tonellato was quoted in Amazon’s release, and he is unlikely to be alone;

“Public Data Sets on AWS will enable me and many of my colleagues to collaborate with each other by sharing our commonly used data sets, research environments and tools. We can set up a controlled environment in minutes, run our computational analysis for a couple of hours, and shut down the environment. Our results are completely repeatable. I only pay for the compute time I use, and more importantly I can spend more time focusing on research, not downloading and setting up computational infrastructure.“

The bigger long-term contribution of this Amazon initiative may actually lie with data that are difficult or impossible to find online today. In a previous existence at the Archaeology Data Service (ADS), for example, my colleagues and I were always being contacted by individuals and organisations with data that they wanted to see online; individuals and organisations that lacked the skills, resources or mandate to mount and maintain the data themselves. How many of those organisations will beat a path to Amazon’s door now… and what sort of resource might we see emerge as a result?

However…

Krishnan concludes his post with a reality-check, commenting;

“this data stored on AWS servers are useful only if the researchers use Amazon EC2 for their computing needs… even if they could tap into it from external platforms, it doesn’t mean much if these public datasets are accessible using some kind of API from their original source itself.”

In other words, much (most? all?) of the advantage Amazon is offering evaporates if developers then have to pull the hosted data off Amazon’s servers and into their own applications running locally or via a competing Cloud provider such as Google.

Although the way in which it is recognised and monetised is finally shifting, data is still valuable, and Amazon (and others) clearly recognise the benefits of enticing users to entrust data to their offering, whilst (almost) imperceptibly making it that little bit more painful to use the data somewhere else.

Kingsley Idehen is quoted as saying,

“The Web’s potential as a globally distributed information space that plugs into disparate databases has never been in question. What has remained unclear is how a federated Web of linked databases would be delivered in a manner consistent with the Web’s core architecture, without compromising its simplicity.”

It is in moving us toward this open vision that Amazon’s offering (although undoubtedly an important step along the way) is ultimately lacking. For that, we may well require the open and linked approach of Semantic Web offerings from companies such as Talis and Kingsley’s OpenLink. These recognise the futility of expecting all data to migrate to a single service provider, whilst still ensuring that those on the ‘inside’ may gain the benefits of proximity on the network, pre-computation of certain indices, etc. Amazon and its services clearly have a place within that emerging ecosystem, but it is a place that they will need to share with others.

The worthwhile philanthropic aspects of Amazon’s announcement apart, the company is certainly doing its part to evangelise the benefits of moving data to the Cloud, and this is to be wholeheartedly welcomed.

CIOs are recognising the benefits of Cloud-based computation, and their resistance to the loss of control implied by individual cost centres’ embracing of SaaS solutions such as Salesforce is diminishing. The proposition of accessing data in the Cloud, at will, is even more profound, and the benefits to be gained require careful and compelling explanation in the face of inevitable fears regarding issues such as data integrity.

Showing everyone the benefits to be gained in sharing disparate public data sets is one more step along the way to widespread acceptance of the value in easing restrictions over access to more sensitive resources.

Related articles by Zemanta

• The evolution of an all encompassing world of clouds
• Amazon Web Services: Bigger Than Amazon
• Public Data Goes on Amazon’s Cloud
• The Amazon Cloud: no longer a mid-Altantic kludge
• Amazon Tries to Lure Scientific Community into the Clouds