Regardless of where you stand on some of the questions of detail with respect to the Linked Data meme, it’s clear that significant enthusiasm is being marshalled behind both the concept and the opportunities that it promises.

Dion Hinchcliffe looks at some of the means by which enterprise data can be more visible on (and useful to) the Web in a ZDNet post this week. The ‘Semantic Web & Linked Data’ are included, and Dion writes;

“By far the most sophisticated and complex of the three approaches to open data presented here, [Linked Data is] highly suitable for certain applications that have rich data sets that need powerful means of processing and consumption. In particular, scientific, technical, medical, mapping, and certain government domains are highly suitable for this approach. It remains unclear if Linked Data will finally trigger the boom in the Semantic Web so use with care. However, definite consideration should be applied, given the potential of the approach to create data sets with extraordinarily high function. Businesses already managing their data with Semantic Web technologies will be the most likely candidates for adoption.”

We’re certainly seeing plenty of talk — and some interesting beginnings — in the Government domain, and organisations such as Thomson Reuters and the BBC are also taking compelling steps around the periphery of their core businesses.

Education offers another interesting set of opportunities, and Jason Ohler’s piece in Educause Quarterly (and a related podcast I recorded with him whilst I was still at Talis) illustrates one view of that opportunity.

Here in the UK, the Joint Information Systems Committee (JISC) is beginning to take note. They first funded a review of ‘Semantic Web Technologies‘ back in 2005, then revisited the topic with ‘Semantic Technologies in Learning and Teaching‘ (and a related workshop in the south of France later this year). I’ll be recording a podcast with the manager of that project, Thanassis Tiropanis, later this month.

JISC have also asked me to conduct a short piece of work to look specifically at the opportunity presented to the Higher Education community by Linked Data, and this work will run over the next few months. I’m certainly keen to learn about concrete examples, and to hear reasoned arguments for and against in order to submit comprehensive findings and recommendations. So if you have something to say, please do get in touch.

Related articles by Zemanta

• Web3 and Enterprise Linked Data. The Middleware Revolution (aldobucchi.com)
• Open government and Linked Data; now it’s time to draft… (johnbreslin.com)
• ReadWriteWeb Interview With Tim Berners-Lee, Part 1: Linked Data (readwriteweb.com)
• Linked Data is Blooming: Why You Should Care (readwriteweb.com)

Thomson Reuters‘ Open Calais team have clearly been busy, with several announcements at the Semantic Technology Conference here in San Jose.

On 15 June the company rolled out version 4.1 of Open Calais, embracing Spanish language content and the notion of ‘social tags;’

“OpenCalais is a great semantic data extraction engine. If you write an article about the relative merits ofPorsche and BMW at the test track in Leipzig, we’ll diligently identify Porsche and BMW as companies and Leipzig as a geography. We’ll create Linked Data URIs to represent these things and open up access to theLinked Data ecosystem so you can enhance your article with other content assets.

But… sometimes you just want a great description. The kind of tags a human would put on the article. Like “Car racing” or “Automobiles”. The kind of tag that would, for example, be very searchable and therefore …. SEO’able (that is definitely is not a word).

In 4.1 we’re introducing OpenCalais Social Tags. Social Tags is our attempt to emulate how a human might tag the document. Social Tags does some fairly sophisticated analysis of your entire document and maps it to a knowledgebase based on Wikipedia and other assets. From that process we generate Social Tags.“

This morning, they followed through with a further pair of announcements. Firstly, CNET has been joined by The Huffington Post, DailyMe and the UK’s Mail Online in integrating Open Calais into their workflow.

One of the more interesting aspects of the earlier CNET announcement was the contribution of data back into the pool;

“CNET joins Thomson Reuters as one of the first commercial media companies to publish core data assets for public, programmatic use on the open semantic Web. CNET will leverage OpenCalais’ connection to the rapidly expanding ‘Linked Data cloud’ to allow its original content — such as tech product reviews on laptops, TVs, smart phones, and digital cameras; news articles and blog posts from its CNET News editorial staff; and parts of its core technology product catalog – to be available for public use.”

It will be interesting to see whether these latest media properties are able and willing to do something similar.

The second element announced today in many ways mirrors Amazon’s recent Import/Export service. Thomson Reuters, too, have recognised that it remains impractical to move large quantities of data over the network, and today announced their ‘Archive Express’ which will process up to 20 million documents off a physical storage device within 24 hours, free of charge.

Related articles by Zemanta

• CNET Partners with Thomson Reuters on Linked Data Initiative (readwriteweb.com)
• Linked Data and the Enterprise: a viable two-way street (cloudofdata.com)
• Semantic technology gains publishing foothold (news.cnet.com)

Harvard University’s Berkman Center for Internet & Society unveiled their Media Cloud research tool today, bringing Semantic Web goodness from Thomson Reuters’ Calais and affordably scalable Cloud oomph from Amazon Web Services together in powering exploration of a fascinating topic.

As the press release notes,

“Media Cloud was conceived by Berkman Fellow Ethan Zuckerman and Berkman Faculty Co-Director Yochai Benkler.  It was inspired by their debate over whether the blogosphere largely echoed traditional media or was instead a source for original news and democratic agenda-setting.

‘While daily newspapers struggle for survival, political, niche and special interest blogs continue to capture consumer interest,’ said Yochai Benkler, Faculty Co-Director of the Berkman Center. ‘In the midst of this upheaval, it is difficult to know where stories begin, who sets the agenda, and how these dramatic changes impact news coverage on the whole.  We created Media Cloud to help researchers and the public get quantitative answers to these challenging questions.’”

The site itself provides more detail, notably,

“Print newspapers are declaring bankruptcy nationwide. High-profile blogs are proliferating. Media companies are exploring new production techniques and business models in a landscape that is increasingly dominated by the Internet. In the midst of this upheaval, it is difficult to know what is actually happening to the shape of our news. Beyond one-off anecdotes or painstaking manual content analysis, there are few ways to examine the emerging news ecosystem.”

By systematically harvesting full-text content from both new and traditional news sources and passing it through Calais for entity extraction, the Berkman team is able to build a coherent and normalised pool of news that they – and others – can begin to interrogate in order to answer pressing questions as to the shifting news landscape. Today’s site is just the beginning, and the team has ambitious plans to increase the number of sources they harvest, to provide ever-richer visualisations on their site, and to expose public APIs to the underlying data in order that third parties can consume it for their own purposes.

Ahead of the launch I spoke with Barak Pridor, CEO of Thomson Reuters‘ ClearForest subsidiary, and Berkman Fellow Stephen Schultze. Both stressed the beta nature of the site, and the fact that the team are still working to optimise a number of areas. That aside, I was impressed by some of the early results.

Barak pointed to the role Calais is playing in enabling the Berkman team to conduct comparisons within and between different news sources, extracting key entities (personal names, companies, places, etc) from the stream of text and normalising the various ways in which we refer to them (IBM, International Business Machines, etc).

Stephen discussed some of the background to the project, and was keen to emphasise the platform nature of their activity; although focussed on a destination web site today, the intention is to expose much of the data via a series of APIs that third parties will be able to consume. In the context of recent comparable moves from the New York Times and Britain’s Guardian, this is clearly to be welcomed and will accelerate the innovation around this incomparable pool of content.

Whilst the site does not currently expose the full richness of the data being harvested, it is already possible to see a number of interesting visualisations.

The maps above, for example, compare the coverage devoted to all the world’s countries by three very different news organisations; the BBC, the Wall Street Journal and FOX News. The apparent non-coverage of Iran is surprising, and suggests an algorithm still in need of tweaking. That apart, it’s actually surprising to note the very similar spread of coverage. European snobbishness about the insularity of US news – and the farcical nature of FOX ‘news’ – may not survive exposure to this data, although we can, of course, reassure ourselves that ‘coverage’ doesn’t always equate to ‘insight,’ ‘analysis’ or ‘truth.’ As more data become available, even those preconceived notions may face an overdue battering.

The system is already capable of addressing more complex questions, and exploring temporal aspects of the ways in which stories break, spread and relate to one another across different media. Stephen shared a number of evolving visualisations that made it possible for me to explore several trends at once, and I look forward to these tools arriving on the site.

The project is directly funded by the Berkman Center at present, and a desire to enrich and stabilise the platform means that significant addition of new news resources may be some months off. I, for one, look forward to seeing more non-US resources such as the UK’s Financial Times and Guardian, or their international equivalents.

The project’s APIs are also being finalised at this point and will be released in due course, along with the source code upon which Media Cloud runs. It will be interesting to see the balance between API access and software download at that point.

Media Cloud represents an excellent example of the uses to which the technologies I cover can be put. It’s not a Semantic Web project. It’s not a Cloud Computing project. It’s an intriguing exploration, that puts those technologies (invisibly) to work in helping to get the job done. That’s the way it should be.

And, as Stephen commented when asked what he was most interested to see,

“the most amazing stuff will be what other folks use it for in asking and answering their own questions.”

Indeed.

Thomson Reuters is already supporting this project by contributing Calais and some technical expertise. I wonder whether some of our more enlightened news organisations might like to help the Berkman get more data in there, faster? As more newspapers close each week, and as we (supposedly) become ever-more insular in the choices we make about the news we consume, there’s a real need to understand the ways in which the media are changing. Many eyes on this data set is one sure way to ensure that the commentary moving forward is informed and informative.

Related articles by Zemanta

• Lawsuit to Determine Fair Use for Blog Links, Headlines (crooksandliars.com)
• Newspapers Suddenly Adapt To Socal Media; Nearly 60 Percent Offer User-Gen Content (paidcontent.org)
• Yes, Twitter is a source of journalism (mathewingram.com)
• Boston Journalists Launch GlobalPost.com, Alternative to Traditional Media’s Shrinking International Coverage (xconomy.com)
• What’s the Best Business Model for Newspapers? (blogs.harvardbusiness.org)

The audience is senior, and easily one of the most engaged I’ve come across for a while. After a ten minute grace period, presenters are pelted with questions and comments, and I’ve only seen a couple get all the way through their prepared slides. It’s refreshing to see attendees making sure they get what they want, rather than listening semi-soporifically to one prepared pitch after another. It should also be good for the speaker, as at least they’re having the audience engage. I say that now. Let’s see if I still think so after my grilling!

My fellow speakers are drawn from a range of backgrounds, and include Russ Daniels, HP’s VP for Cloud Services Strategy, Greg Papadopoulos, Sun’s CTO, Vishal Sikka, SAP’s CTO, and Doug Cutting, Yahoo!’s Project Lead on Hadoop.

Related articles by Zemanta

• TED2009: Tim Berners-Lee (boingboing.net)
• HP Confines the Cloud for Enterprises (gigaom.com)
• Linked Data and the Enterprise: a viable two-way street (cloudofdata.com)

I am travelling to San Diego next week to speak about the importance of evolving Enterprise attitudes to data. Borrowing some nice turns of phrase from Sir Tim Berners-Lee‘s recent TED talk and JP Rangaswami‘s keynote to Powered by Cloud, amongst other things I’ll be suggesting that they ‘stop hugging their data’ and move ‘from data centre to data centric.’

The Linked Data initiative, which began in March of 2007 as a community project supported by W3C‘s Semantic Web Education & Outreach (SWEO) Interest Group (of which I was a member), has been a huge success. Described by Berners-Lee as ‘the Web done right,’ the notion of Linked Data rests upon the acceptance of four simple principles, yet opens the door to previously unanticipated re-use of data scattered across the Web.

The most rapid adoption has, unsurprisingly, been seen in terms of liberally licensed data already visible on the Web in some form. DBpedia, for example, is a community effort to extract structured information from Wikipedia and expose the individual facts for use across the Web. There have also been examples — as always justified by hacker mentality, ‘academic freedom,’ the imprimatur of ‘research,’ or the expectation that the perpetrators are ‘too small’ to be noticed — in which data have been appropriated to the cause without due care and attention to the rights of the data owner, but these isolated cases should certainly not detract from the value of the broader effort.

Public Interest data from organisations such as the BBC has also begun to appear in the ‘Linked Data Cloud‘ (click on individual data sets for more),  and the frequency and strength of reciprocal links between participating resources grows rapidly.

Enterprise data is effectively invisible to this Cloud, which brings me back to Andreas’ first post. In it, he asks;

“Since the [Linked Data] cloud is kind of the basic infrastructure which drives the whole process – this layer should remain a freely accessible one. But how could new business models be built on top of it (and constantly spend money on maintaining and extending the underlying infrastructure)?

Where could enterprises start using Linked Data? Only by retrieving data from the ‘outside’ and mash it up with the ‘inside’ – only one way?”

I can certainly see cases in which cautious corporates will be willing to consume without contributing in return, and there’s clearly work to do in demonstrating the value that they could gain from more balanced participation; participation that should never mean unwillingly ‘giving away’ competitive advantage or sensitive data.

We have an annoying tendency to view data in our databases as an indivisible mass, vigorously and unthinkingly applying the same (expensive) protections to an uninteresting and low-value factoid of underlying context as we do to the core attributes of our next big lead.

Andreas concludes this post by suggesting something very similar to JP Rangaswami’s notion of ‘data centric’;

“Information has no ‘place’ anymore, energy can’t be shipped around the world. We should rethink the meaning of a ‘data store’ and information will flow without flooding us. Linked Data might become the essence.”

Andreas’ second post followed after he’d listened to the most recent episode of the Semantic Web Gang, which I Chair. During the show, recorded last month, we discussed the latest release from Thomson Reuters‘ Open Calais activity, which sees it embrace Linked Data’s principles whilst continuing to run and grow a viable global business.

Andreas extrapolates from the conversation to suggest that a viable business model for the data-curating Enterprise might be to expose timely and accurate enrichments to the Linked Data ecosystem; enrichments that customers might pay a premium to access more quickly or in more convenient forms than are available for free. He also sees a market for application builders that optimise the flow of information, and both of these are certainly possible.

The Linked Data — the Data Web — opportunity is far greater, though, and too little attention is being devoted to it by Linked Data’s advocates as they concentrate their efforts on big public datasets of the sort Berners-Lee discussed in Long Beach last week. Big public data sets are important, and Berners-Lee is right to suggest that more Open and Linked access to the outputs of scholarship will help in our efforts to tackle many of the world’s ills. There’s as much value locked up inside our commercial enterprises too, though, and yet the rationale that will ultimately lead to us unlocking this is quite different.

It is that rationale which we need to get right, almost certainly without mentioning ‘RDF’, ‘Semantic Web,’ or even ‘Open.’

And if you’re in Southern California next week too, why not come and say ‘Hi’…?

‘Sunrise over San Diego‘ image © Alon Banks, 2007

Related articles by Zemanta

• TED2009: Tim Berners-Lee (boingboing.net)
• Linked data will change health care (ourownsystem.com)
• The Semantic Web in Action (sciam.com)
• Tim Berners-Lee: Linked data (thenumerati.net)
• Building coherence at bbc.co.uk (blogs.talis.com)
• Calais 4.0 Released: Linked Data Meets the Commercial Web (readwriteweb.com)
• Calais Release 4 and the Linking Data cloud… (ivanherman.wordpress.com)

On the face of it, there surely can’t be many places more ready for the Semantic Web than the world’s great museums. Vast quantities of richly structured data describing visually compelling objects that people already flock to see? A raison d’etre that is about sharing information and enhancing understanding? A tradition, stretching back several hundred years, of describing things clearly?

If my conversation with Koven Smith of New York City’s Metropolitan Museum of Art is any indication, though, the truth would appear rather different.

He talks, depressingly but all too familiarly, of vendor lock-in, competing institutional priorities, and a tradition of remarkable insularity within institutions that really should be more open.

Even in the face of this, though, Koven is able to describe work at the Met to leverage third party tools such as Open Calais and Semantic MediaWiki in enriching possible interactions with the museum’s institutional memory.

Have a listen, and see what you think.

I am aware of similar work at Amsterdam’s Rijksmuseum and behind the European Commission‘s struggling Europeana, and should try to secure similar conversations with them. Is anyone aware of other examples?

Related articles by Zemanta

• How will we interact with all this data?