Richard Wallis

Our conversation is also an excuse for an experiment. I have been producing audio-only podcasts here and elsewhere for a number of years, but have always tended to avoid producing video. It’s more effort, it requires more bandwidth at both ends of the conversation, and I’ve never really been convinced that it adds very much to a conversation between two people. Anecdotal evidence would also suggest that my current podcasts are consumed in environments where video would not work; washing dishes, walking dogs, and sitting on buses.

However, rather than just continue to presume that my biases are correct, I’ve decided to give video a try. Richard kindly agreed to participate, and the result is available on YouTube and embedded here.

An audio-only version is also available for download if you prefer. The introductory remarks in this version are slightly different to those on the video, as they come straight from the original conversation.

It’s perhaps unfair to draw too many conclusions from this first attempt, but a few things are immediately apparent. The whole process takes an awful lot longer. The files are larger, so processing and uploading times increase 2-3 fold. Uploading a separate audio file also takes a bit of time. Simply dumping the Skype recording into iMovie worked just fine… but I’ve (so far) not managed to find any way to balance the audio levels. Garageband lets me do this with my audio-only podcasts, but iMovie doesn’t seem to, so Richard’s side of the conversation comes across as quite a bit louder than mine.

Having done one, I’m still not convinced that the video adds anything to the conversation. But what do you think? If you’ve listened to any of my podcasts, please take a moment to complete the short survey over at SurveyMonkey. Your responses will help me to decide where to go next.

Many thanks.

Image via CrunchBase

Chris Hathaway sees basic location information scattered across the websites of hundreds — or thousands — of coffee shop chains, hotel groups, and fast food joints, but argues that it’s almost impossible to do anything more sophisticated with the data than find your closest Starbucks. His company, AggData, is attempting to fill what he sees as a gap in the market; scraping addresses and other facts off company websites to create simple files of store locations that can then be enriched with coordinate data and sold.

Customers for this data include competitors, market researchers, consultants, and even the companies themselves; as is so often the case, it can be easier to buy data on store locations from a third party than to find the authoritative sources within your own organisation. AggData is strongest in the US today, but also offers a growing body of data for other countries. Although the data files are structurally simple, Chris sees plenty of opportunity to continue collecting and selling data to a growing community of customers.

Unlike Factual, which was the focus of last week’s podcast, AggData is not currently interested in combining data from different sources. Customers download separate files on the locations of Starbucks, Peets and Tim Hortons, and not a single aggregated set of coffee shop locations. The AggData model is also predicated upon using their own scripts to extract data from third party sites; asked if he would accept a file of WalMart store locations supplied by WalMart, Hathaway explained why he would — and does — decline.

Have a listen to learn more about AggData, and to hear Chris’ perspectives on the potential role of semantic technologies in making his job easier. And check back on Thursday for the next podcast in the series; Hjálmar Gíslason of DataMarket.com.

Following up on a blog post that I wrote at the start of 2012, this is the second in a series of podcasts with key stakeholders in the emerging category of Data Markets. Other conversations, all of which will be published here, have been scheduled with BuzzData, DataMarket.com, Factual, Infochimps, Kasabi, and Microsoft. I am still adding conversations to the series, and intend to talk with more companies and with analysts and investors with insight to share. 

Related articles

• Nurturing the market for Data Markets (cloudofdata.com)
• Data Market Chat: the podcasts are a-coming… (cloudofdata.com)
• Data Market Chat: Tyler Bell discusses Factual (cloudofdata.com)

Image of Stephen Wolfram via Wikipedia

British-born computer scientist Stephen Wolfram sees ongoing efforts to extend the Internet’s top-level domains (TLDs) beyond the familiar .com, .org, .uk etc as an opportunity to raise the profile of machine-readable data. In a blog post published yesterday, he argues that a new .data domain would increase “exposure of data on the internet—and [provide] added impetus for organizations to expose data in a way that can efficiently be found and accessed.” Whilst wholly in favour of Wolfram’s stated aim, I can’t help feeling that his suggested solution is at best unnecessary and at worst a worrying segregration of data from the ‘proper’ web that everyone else will continue to exploit.

Back in June of last year, the body responsible for coordinating the global domain name system approved a plan to permit new top-level domains (the letters after the final dot in an internet address — the .com in cloudofdata.com, the .uk in bbc.co.uk, the .edu in harvard.edu). Until recently, these top-level domains have been tightly controlled, with a small set of generic domains (.edu, .gov, .mil, .org, etc), a larger set of country domains (.uk, .fi, .nz, etc) and one or two others such as .eu. From tomorrow, anyone with $185,000 will be able to submit a proposal to create and manage a new top level domain, and it’s possible that there could eventually be thousands of them. Wolfram is keen to ensure that data doesn’t miss out on the ‘opportunity.’

As Wolfram himself recognises, there is already an awful lot of machine-readable data on the web. Some of it sits embedded within the web pages that humans read, with specially formatted code waiting to be triggered by the calendars, the address books, or the browser plugins of site visitors. Some of it is packaged up in data files, offered for download. And some of it waits inside a database, ready to be delivered in response to an API call or a query typed into a web form.

There is a growing enthusiasm for exposing this data for reuse. Government transparency agendas have driven public sector data sites like data.gov.uk and data.gov. Similarly, efforts such as data.open.ac.uk and data.southampton.ac.uk see universities beginning to consciously collect data sets together and offer them up for reuse. Similar efforts in the commercial world are less easy to point to, but that reticence has nothing whatsoever to do with the lack of a ford.data, boeing.data, ge.data or astrazeneca.data domain!

In some ways, the convention for gathering significant chunks of data on a data.xxx.yyy site echoes Wolfram’s intention, but with a number of advantages. Data without context is far less valuable than data with context. Much of that context may be inferred from the domain in which the data lives, with data delivered from a .gov or .edu (or .gov.uk or .ac.uk) site perhaps interpreted differently to data hosted on .com, .biz, or .xxx. Southampton University, the Open University, and the US Federal Government are able to gather data up and make it available for download via their existing data. sites if they choose. This offers human visitors to their sites a degree of convenience, whilst retaining the power and brand attributes of their existing domain. Gov.data, gov.uk.data, open.ac.uk.data, southampton.ac.uk.data, though? All are messy, in ways that Wolfram’s own wolfram.data would admittedly not be, and all are simply additional registrations that the institutions would have to pay for in order to stop someone else grabbing the domain.

At the end of the day, the machines don’t actually care. The existing data.open.ac.uk-type sites are human conveniences, not machine enablers. The computers, and the software they run, are quite capable of crawling the public web and finding accessible data wherever it lies on a site. There are plenty of reasons to continue embedding little snippets of data inside human readable web pages, regardless of whether you have a data.wolfram.com or a wolfram.data site. Content negotiation is becoming increasingly capable, such that there really is no need for what Wolfram calls a ‘parallel construct to the ordinary web’ at all. A human being arriving at a web site sees human readable content, whilst various software tools would automatically be presented with very different data or functions, optimised to their capabilities and requirements.

By all means, let us show the curious some of the existing techniques that work in making data more easily accessible. By all means, let us identify the gaps, the issues, the problems (none of which a new TLD even begins to address). Yes, let us definitely and unambiguously set about “highlighting the exposure of data on the internet—and providing added impetus for organizations to expose data in a way that can efficiently be found and accessed.”

But please, let us not be distracted by the false hope that adding yet another TLD to the babel that ICANN is about to unleash can do anything more than consign data to some online ghetto, wallowing unwanted, unloved and unused as companies and their customers lavish love, attention, and clicks upon the .com domain over on the ‘proper’ web.

Thanks to Raphaël Troncy, whose tweet first drew the story to my attention.

Related articles

• Is It Time For Computers To Have Their Own .Data Domains? (techcrunch.com)
• ICANN Pushes Ahead With January 12 Launch For New Top-Level Domains (wired.com)
• The biggest change in DNS since Dot-Com (wired.com)

Image via Wikipedia

For real-world applications of Linked Data and the Semantic Web, the long-running Semantic Technology Conference is hard to beat. For getting a real handle on the Cloud Computing landscape, GigaOM‘s Structure Conference is also a leading light.

Working across both areas as I do, these events tend to figure prominently in my calendar for the year*. Last year, both took place in San Francisco during the same week. I tried to attend both, and therefore succeeded in spending most of my week in cabs, shuttling between meetings at the two venues. I saw very few sessions that I wasn’t personally involved in, and the experience wasn’t a huge success.

This year the conference organisers have taken pity on me, and moved their events to opposite ends of June. The Semantic Technology Conference is up first, back at the Hilton Union Square from 5-9 June. Structure follows, returning to the Mission Bay Conference Centre on 22 and 23 June.

I’ll be attending both, and probably doing various official things during each event. At the moment, the only thing we’ve definitely nailed down is a special live appearance by The Semantic Link crew on the evening of 5 June; we’ll be taking a look at the highlights expected for the conference, offering some tips for those new to the event and its multitude of parallel sessions, and generally bringing our usual podcast chatter to the stage.

So if you’re in town around the time of either event, do get in touch; it promises to be an interesting month.

* Disclosure: WebMediaBrands pay me to host the monthly Semantic Link podcast, and to write a monthly column on SemanticWeb.com. GigaOM pay me to curate the Infrastructure/ Cloud Computing channel on their Pro site. I attended and participated in both of these events before that was the case, and still would today without the contractual relationship.

Related articles

• GigaOM Structure 2011 (datacenterknowledge.com)
• Semantic Technology Conference (fakeiitian.com)

Krista Thomas was responsible for marketing at Thomson Reuters‘ semantic technology success story, Open Calais, and is now VP Marketing at Los Angeles startup Ad.ly.

Scott Brinker is President & CTO at ion interactive, and writes the ChiefMarTec blog.

During the conversation, regulars and guests discuss the way in which ‘semantic technology’ solutions are being successfully marketed to potential beneficiaries.

Have a listen, and learn that pitches to non-semantic technology enthusiasts beginning “We’ll convert all your data to RDF, leverage the Semantic Web, and construct an OWL-based ontology” are probably doomed to failure.

The whole team is present once more, and we start the show discussing the implications of Drupal 7 and its newly formalised RDFa-publishing capabilities. Unlike regular semantic technology solutions, which someone has to consciously procure as a Semantic Technology solution, Drupal is first and foremost a (popular, free) Content Management System; the semantic smarts come for free, and therefore reach a massive new audience.

From there, we move into a broader discussion of the ways in which semantic projects take root within organisations.

Have a listen, and let us know what you think. As ever, suggestions for future topics or guests are always welcome.

@theSemanticLink is the show’s Twitter id, which we will use to invite questions ahead of shows on particular topics. If you’d like to ask questions, comment on shows, hear about new episodes as they’re released, etc, please do sign up to follow.

Related articles

• The Semantic Link is open (cloudofdata.com)
• Drupal 7 Released, With Improved UI and Semantic Technology (nytimes.com)
• House.gov moves to Drupal (radar.oreilly.com)
• Dries Buytaert Explains How Drupal Gardens SaaS Can Contribute to Semantic Web Momentum – semanticweb.com (semanticweb.com)
• Drupal 7 dives into machine-readable web (go.theregister.com)

We recorded the first episode earlier this week and it’s now online, along with an introduction to the series from myself.

Please do have a listen, and let me know what you think whilst I work out how to automate adding these to my sidebar on this site…

I look forward to lining up some great conversations – and guests – in the year ahead. We will record and publish regularly each month. Where we can agree a topic well ahead of the recording date, I’ll let you know via the @TheSemanticLink twitter account, and invite questions for me to throw at both regulars and guests.

If there are particular topics you’d like to see covered, particular guests you’d like us to secure, or other comments you’d like to make, please do let me know…

Related articles

• Top 10 Semantic Web Products of 2010 (readwriteweb.com)
• Looking forward to The Semantic Link podcast on SemanticWeb.com (cloudofdata.com)

‘Big Data‘ is currently capturing the imagination, attracting hype, investment and ambitious startups in almost equal measure. Kim and Eric Norlin’s excellent Defrag and Glue events have gained big-name company, with O’Reilly‘s Strata and GigaOM‘s Structure both set to arrive in the first quarter of 2011. Venture firms like IA Ventures have emerged, specifically targeted at finding, funding, and profiting from the big Big Data idea. Giants of the web from Yahoo! and Amazon to Twitter and Facebook solve their own Big Data problems in very different ways, contributing valuable code and experience to the community whilst simultaneously diluting focus and adding to the cacophony.

Flippantly reckoned by many to be ‘anything that requires more than a single machine to run,’ the Big Data reality remains somewhat harder to pin down. To those seeking routine business insight, that mammoth Excel spreadsheet they laboriously query overnight at the end of each month might quite justifiably be thought of as ‘Big.’ At the other end of the scale, data wizards scorn anything that doesn’t require a room full of servers, a mountain of empty pizza boxes, and the careful construction of a bespoke data ingest, management and querying system atop the most bare-bones version of the Linux kernel they can find. Somewhere between the two, a growing mass of cheaply gathered data holds out the promise of invaluable insight. Remote sensors, web clickstreams, social graph interactions, purchaser (and non-purchaser) behaviours. All these, and more, have much to tell planners, builders, makers, sellers, and buyers. If only we could formulate the right questions. If only we could devise the right sampling strategies. If only we had big enough machines to ask lots of questions using lots of sampling strategies. If only we had big enough machines to not bother sampling at all.

On the hardware side of things, even humble domestic laptops typically ship with at least two cores these days; two separate little computers ready to do the data processor’s bidding. Four, eight, sixteen and more cores are not far behind, but mainstream software products typically fail to exploit anything more than a single core. Push Excel as hard as you like, and it won’t do more than take one of your computer’s multiple cores to the max. On that 12-core Mac Pro you persuaded the boss to buy, only one core will be hard at work on your data. Twitter, Mail, YouTube, and ripping DVDs  will each be giving other cores a little light exercise whilst others sit idly by, waiting for the arrival of operating systems and applications capable of exploiting multi-core power. The same is true as jobs grow and move to run across multiple machines, whether under your desk, in your data centre, or out in the Cloud. Those big datasets need to be carved up and shared amongst the available computers before any analysis takes place. You’re typically not accessing a ‘big computer in the Cloud’ at all… but lots of relatively small (commodity) computers, and it takes careful planning and smart software to manage the division and recombination of those jobs in a cost-effective manner. Projects such as Joseph Hellerstein‘s Berkeley Orders of Magnitude (BOOM) begin to demonstrate some of the potential for working natively with multiple processors, but there’s a long way to go before those advances reach the mainstream.

Hadoop, Cassandra, MapReduce, Dynamo, Voldemort. These, and more, are solutions developed by the likes of Yahoo!, Facebook, Google, Amazon and LinkedIn to tackle the influx of data that each faced – and for which each had failed to find an existing solution. Hadoop, with the addition of Cloudera‘s commercial polish, is rapidly emerging as the front runner for an off the shelf Big Data solution, but all of these tools remain rather narrow in their abilities. Find the type of data or the nature of query for which each of these was built and its performance will be unbeatable, but we are a very long way from Big Data’s equivalent of the jack-of-all-trades SQL-powered relational database of old.

And there, for many enterprises, lies the problem. Useful Google searches require the crawler, index and UI to do a relatively small number of essentially similar tasks, very quickly, very cost-effectively, and at massive scale. Focus on that finite set of problems, and you build a solution that delivers the experience we’ve all come to know. Each type of data manipulation or analysis requires a different tool, differently optimised, with the inevitable result that a typically diverse organisation may require a plethora of Big Data tools to get their work done. Or they might just continue to muddle along with Oracle or mySQL, churning inefficiently through their data analysis jobs for interminably long periods of time. These relational database tools are understood, they are mature, and they get the job done. Except in the most data-intensive industries, they have a market presence that will be difficult to disrupt.

The Big Data space is seeing remarkable innovation, but there is a long way to go in order to lift it out of the domain of the technically proficient specialist and place it on desktops across the organisation. As IA Ventures’ Brad Gillespie notes, “Excel is where the world’s data lives… [and] Big Data has to get to that place… so that a CMO can leverage it directly.”

And in all of this fervent of innovation, to return to the title of the post, it strikes me that Big Data is becoming disconnected from the fabric of the web itself. Oh, much of the data certainly comes from the Web, and a lot of it might even be queried on the Web after processing. But, somewhere along the line, the linkedness of the Web has either been forgotten or ignored. That rich set of connections, interconnections and associations has been reduced to a table, an index, or a (large) set of key-value pairs. And in the process, something fundamental has gone away.

This is enough for now, though. Looking more closely at different Big Data approaches, and exploring the potential for re-introducing the Web must wait for future posts.

Related articles

• Cloudera raises $25M to help deal with the enterprise data deluge (venturebeat.com)
• Strata Week: Building data startups (radar.oreilly.com)
• Big Data and a Critique of Geek Culture (readwriteweb.com)
• Big Data and NoSQL March to the Enterprise (nytimes.com)
• Does ‘big data’ equal big opportunity for storage vendors? (news.cnet.com)
• New York Times Event Shows the Promise of Big Data (programmableweb.com)
• Overwhelmed Executives Still Crave Big Data, Says Survey (readwriteweb.com)

Almost exactly two years ago, I announced my intention to leave Talis and strike out on my own. A year ago, the final piece of that transition concluded when we published the last episode of the Semantic Web Gang; a round table podcast that I began inside Talis and subsequently produced with their sponsorship.

Fast forward to the end of 2010, and it’s time to revisit the value that a monthly round table conversation offered to the community. Talking with friends, customers and contacts, many of them pointed to the useful role that something like the round table discussion format plays in cutting through marketing hype, community groupthink and competitor obfuscation. The old Semantic Web Gang did that, and it did it because it comprised a great group of contributors who knew their stuff.

Working with Eric Franzon at semanticweb.com we are about to recreate some of that value by launching The Semantic Link, and I am really pleased with the panel of regulars that Eric and I have signed up; a completely new group of contributors who also know their stuff.

Our first episode will be recorded in early December, and should be pushed out via semanticweb.com (and iTunes, no doubt) before Christmas. We’ll then settle down to a regular cycle, recording and publishing each month, dissecting whichever trends, stories, initiatives and companies happen to be topical. I’m also keen to get guests on the show from time to time, especially where they’re involved in something truly interesting and newsworthy.

If you have ideas for topics you would like to hear us dissect, please do let me know.

Bill’s slides are here, and mine are embedded below.

Thanks to Paul for the invite, and to the audience for braving an intermittently (very) wet Manchester evening to spend a few hours discussing Linked Data, the Semantic Web, and related topics; it was fun.

Related articles by Zemanta

• Google Acquires Metaweb To Make Search Smarter (techcrunch.com)
• Tim Berners-Lee Doesnt Seem to Think Linked Data Requires RDF (ironick.typepad.com)
• CubicWeb – The Semantic Web is a Construction Game!: OSCON 2010, The O’Reilly Open Source Convention – O’Reilly Conferences, July 19 – 23, 2010, Portland, OR (oscon.com)
• 15 sessions on marketing and the semantic web (chiefmartec.com)