In my latest piece for GigaOM Pro, published today, I take a look at some of the ways in which the elastic nature of Cloud Computing is being exploited in order to support the increasingly rich interplay between on-screen and online content.

As more of us watch television whilst also able to get online with a laptop, tablet or smartphone, the opportunities for further interaction – and the prospect of extreme spikes in traffic within seconds of an on-screen prompt – grow by the day. There are some interesting examples of early success here, but it remains an opportunity that is still a very long way from being fully understood or exploited.

Thanks to Tom Lounibos at SOASTA, Andy Parker at Channel 4, and everyone else who took the time to speak with me as I prepared the piece.

Image ‘138/365: It’s not Rocket Science FUTAB‘ by Christopher Thomas, shared on Flickr with a Creative Commons license.

Related articles by Zemanta

• Go Daddy Ad Drives Huge Traffic Spike (datacenterknowledge.com)
• Monday Morning Marketing Quarterback: Which Superbowl Ads Scored On The Web? (techcrunch.com)
• How MySpace Tested Their Live Site with 1 Million Concurrent Users (highscalability.com)
• IBM Testing Cloud Set for Production (datacenterknowledge.com)

I shared some of my views on this site soon after the event, but think you’ll like the range of opinions and insights from regular Gang members, participants in our audience, and conference organiser Tony Shaw. Have a listen, and see what you think.

As well as generating our regular audio podcast, the event organisers were kind enough to also video the session, and publish the video on their site. I’ve also embedded it, below.

In fact, there’s a nice set of resources from the conference taking shape over on the Semantic Universe site; take a look and see video from the Linked Data Panel (another of the sessions I moderated), the keynote sessions and more.

Related articles by Zemanta

• Tom Gruber talks about Siri, the Virtual Personal Assistant (cloudofdata.com)
• Don’t Miss: Semantic Web Gang, Web 3.0 Conference, Semantic Technology Conference (semanticsincorporated.com)
• Bing is not alone; similar techniques alive and well in existing vertical search (blogs.zdnet.com)

TripIt is one of those web applications upon which I have really come to rely. Like Tungle, it sets about reducing the pain of dealing with the admin behind a boring, repetitive, frustrating yet necessary part of my work.

For Tungle, as I’ve said before, that task is meeting scheduling. For TripIt, it’s organising and tracking the various elements of my travel arrangements. Not only does TripIt reduce the hassle, but it does this in an understated fashion that doesn’t impact adversely upon my workflow; and in the process it adds value so that I end up with less pain, less wasted time… and more valuable information.

I was delighted to have the opportunity to speak recently with TripIt co-founder and VP of Engineering, Andy Denmark. The result has just been released as one of my podcasts.

Production of this podcast was supported by Talis, and show notes are available on their Nodalities blog.

By simply forwarding all those automated booking confirmation messages from airlines, hotels, rail companies and car rental sites to TripIt, the site builds an itinerary and makes it available for synchronisation to your calendar. It does all that in less time than it would take to enter the details yourself, whilst also storing a copy online, making it available for sharing with your network via LinkedIn, a blog plugin, etc, and automatically adding additional information such as the weather forecast at your destination, directions from the car rental location to your hotel, and more.

Personally, I find that the biggest advantage is simply building the itinerary and getting it into iCal quickly, accurately and painlessly. It’s also useful to be able to share flight arrival times, hotel phone numbers etc with family, should they ever need them.

TripIt recently introduced a premium service with some additional features, which I have yet to try.

Behind the scenes, TripIt is drawing upon a wealth of structured data scattered across the Web. It is also doing a lot, internally, to add structure to the free text of those booking emails, and sometimes it is more successful at this than others.

With Yahoo! SearchMonkey and recent announcements from Google likely to drive an explosion in more structured data on the Web, TripIt perhaps shows us a small glimpse of what might become commonplace; dedicated vertical apps mining our online presence to enrich, add value, and make our lives easier in small but important ways. I’d argue that building the next generation of these applications will be even easier, as increased public scrutiny leads to cleaner, richer data with which to work, and ever-more APIs from the Web’s Platform companies sees application builders increasingly able to stand, with ease, upon the shoulders of those giants. I look forward to finding out.

Related articles by Zemanta

• Google embraces exercise in Semantics (news.cnet.com)
• RWW Live: Online Travel (readwriteweb.com)

Image via Wikipedia

Two separate pieces of news came my way during the night, and although both were written about elsewhere whilst those of us on this side of the Atlantic slept, they remain worthy of mention; both in their own right and because of the wider trend of which they are part.

First, Cloud Computing provider 3Tera announced their AppStore;

“the first marketplace for cloud components where enterprise users, software vendors and datacenter experts can exchange production-ready, scalable and highly available cloud components on a pay-per-use basis.”

You can listen to my recent podcast with 3Tera’s Bert Armijo, here.

Second, Semantic Technology pioneer AdaptiveBlue‘s CEO (and Semantic Web Gang regular) Alex Iskold sent an email to point me at their launch of a new Glue API;

“This new API taps into Glue’s databases and semantic recognition engine enabling fun & useful applications about people and things.”

Both companies recognise that there is a far greater pool of talent outside their employ than inside, and both are seeking to place themselves and their technology at the centre of a scalable ecosystem rather than perpetuating the old fashioned model of supplying solutions. Both recognise, too, that the ‘sticky’ destination site is becoming increasingly irrelevant to many of our online behaviours. We want and need functionality, community, reliability and more; but we want it on our terms, delivered in real time at the point of need.

By seeking to build and mediate a critical mass of third party applications, 3Tera is repeating a formula successfully demonstrated by the likes of Apple, Salesforce and others. Partners and developers deliver far more working code than 3Tera’s own developers could manage, and those partners are incentivised to bring their own customer base along with them. 3Tera benefits every time one of these partners makes a sale — for negligible effort on 3Tera’s part — and has an easy route to new customers of its own via its partners. In time, 3Tera’s own AppLogic may even come to increasingly be perceived as no more than an on-ramp to the AppStore’s riches.

AdaptiveBlue’s API extends Glue in an obvious direction, adding to site-independent strengths upon which I have remarked previously.

The world is changing. By embracing the power of networks (both technological and social) and putting others to work on your behalf, companies are increasingly able to punch far beyond their own weight. It is ever-more feasible for small, agile, responsive and engaged organisations to draw upon the resources of others to mutual benefit. As Newton once wrote,

“If I have seen further it is only by standing on the shoulders of giants.”

Related articles by Zemanta

• 3Tera to Support AppLogic with New AppStore, Now Seeking Cloudware Vendors (readwriteweb.com)
• Glue Gets Stickier With Conversations and Recommendations (readwriteweb.com)
• How the Cloud Will Disrupt the IT Status Quo (gigaom.com)
• The open, social web (factoryjoe.com)

Much has been written about growing Enterprise use of social media (usually Twitter, these days) to successfully track and mitigate customer complaint. Many have been quick to spot that the disproportionately high cost of satisfying (or, more cynically, silencing) these early adopters is unlikely to scale effectively as an increasingly large cohort of customers move onto these services, and it must remain an open question as to whether ComcastCares and its peers can survive any move to the mainstream in recognisable form.

It appears, though, that Enterprise engagement in the social sphere changes the game far more significantly than merely enabling a select few twitterati to jump the Customer Support queue, and that this change is worth effort and investment in order to ensure that it does scale. What’s actually happening is that a relationship is being enabled between a brand and what Seth Godin might recognise as its tribe; a relationship in which interactions are no longer driven predominantly by the desire to seek redress. Rather than only raising those issues serious enough for us to have written letters or endured telephone muzak in the past, we now comment on issues at the periphery of a brand. Collectively, we’ve moved from simply complaining about the worst failures of companies, their products and their employees, toward emitting an impressive stream of FYIs. Individually insignificant, and possibly unimportant, together these light touches on and around a brand build into an ever-changing and valuable commentary that brands and the corporations they front would do well to take notice of. The minor niggles about an otherwise exemplary service, the human touches that made us smile, the odd inconsistencies in a polished persona; none are enough to make us pick up the phone, but we comment upon them endlessly in Twitter, Facebook, FriendFeed and elsewhere, and by tapping into this fundamentally honest stream of consciousness there is much for those about whom we comment to learn. Good companies probably already know about fundamental failings in a product long before their customer support operation melts down under the weight of complaints or their quarterly sales targets are seriously under-achieved. Do they have as good a handle on the things we love? Do they have a clue about the minor gripes of customers outside their pre-launch polling groups? Do they know about the gut reaction to a colour, a touch, a smell, or a careless word that persuaded a likely prospect to buy a technically or aesthetically inferior product from the competition instead? All this and more is there for the taking in the stream of online chatter freely directed their way.

Related articles by Zemanta

• Talk With Me, Not At Me (regulargeek.com)
• Is Your Goal Growth? Empower your Employees with Social Media. (kylelacy.com)
• Max Gladwell: 10 Ways to Change the World Through Social Media (huffingtonpost.com)
• The Bogus RSS Debate (cloudave.com)
• RSS is (still) alive (socialmediatoday.com)
• The Conversation Index (briansolis.com)

Amid mounting international concern that the guidance lasers aboard Jeff Bezos‘ new Floating Amazon Cloud Environment would interfere with Rudolph‘s sense of direction, sources close to the Amazon Web Services team tell me that they’ve been forced to alter priorities and switch attention to an early release of the next product on their roadmap.

Today sees the release of Amazon‘s latest web service; the Hadoop-powered Elastic MapReduce;

“Using Amazon Elastic MapReduce, you can instantly provision as much or as little capacity as you like to perform data-intensive tasks for applications such as web indexing, data mining, log file analysis, machine learning, financial analysis, scientific simulation, and bioinformatics research. Amazon Elastic MapReduce lets you focus on crunching or analyzing your data without having to worry about time-consuming set-up, management or tuning of Hadoop clusters or the compute capacity upon which they sit.”

The company’s press release quotes VP for Product Management & Developer Relations, Adam Selipsky, who notes;

“Some researchers and developers already run Hadoop on Amazon EC2, and many of them have asked for even simpler tools for large-scale data analysis. Amazon Elastic MapReduce makes crunching in the cloud much easier as it dramatically reduces the time, effort, complexity and cost of performing data-intensive tasks.”

MapReduce was brought to prominence by Google, and is one of the principal techniques at that company’s disposal in enabling them to break massive data sets into manageable chunks suitable for cost-effective processing on the commodity hardware for which they are known. The abstract for a Google research paper on the topic outlines the value proposition reasonably succinctly;

“MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key. Many real world tasks are expressible in this model, as shown in the paper.

Programs written in this functional style are automatically parallelized and executed on a large cluster of commodity machines. The run-time system takes care of the details of partitioning the input data, scheduling the program’s execution across a set of machines, handling machine failures, and managing the required inter-machine communication. This allows programmers without any experience with parallel and distributed systems to easily utilize the resources of a large distributed system.

Our implementation of MapReduce runs on a large cluster of commodity machines and is highly scalable: a typical MapReduce computation processes many terabytes of data on thousands of machines. Programmers find the system easy to use: hundreds of MapReduce programs have been implemented and upwards of one thousand MapReduce jobs are executed on Google’s clusters every day.“

Hadoop is a Yahoo!-nurtured Open Source equivalent to Google’s MapReduce, managed as a project of the Apache Software Foundation, and reputedly scalable to handle many petabytes of data distributed across thousands of CPUs.

As Adam noted in the press release, customers (such as the New York Times and Netflix) are already using Hadoop on Amazon’s Web Services. Today’s announcement makes it easier to cost-effectively and transparently commission (and decommission) the required compute resources. This is the ‘elasticity’ referred to in the new service’s name, and is an increasingly important aspect of the current generation of Cloud-based compute services; much of the economic value proposition lies in only using (and therefore paying for) the resources you actually need to complete a task. If demand increases, the number of (virtual) machines available should rapidly increase to cope, and they should shut back down just as rapidly when the demand passes;

“Amazon Elastic MapReduce enables you to use as many or as few compute instances running Hadoop as you want. You can commission one, hundreds, or even thousands of instances to process gigabytes, terabytes, or even petabytes of data. And, you can run as many job flows concurrently as you wish. You can instantly spin up large Hadoop job flows which will start processing within minutes, not hours or days. When your job flow completes, unless you specify otherwise, the service automatically tears down your instances.“

Elastic MapReduce is currently available only for data centres in Amazon’s US region (so non-US customers can use the service; they just have to be able/willing to transfer the data beyond their borders), and is priced in addition to existing EC2 instances with Elastic MapReduce on a $US0.10 per hour ‘small’ instance costing a further $US0.015 per hour (yes, 1 and a half cents per hour) and on a $US0.80 per hour ‘extra large’ instance costing a further $US0.12 per hour.

Elastic MapReduce is another nice example of slow, incremental improvement to Amazon’s core Web Services offer.

It remains to be seen, as developers get down to using it for real, whether it’s pitched as a low-end disruptor that simply rounds out another piece of the emerging AWS whole, or if it’s a viable competitor in its own right to the recently announced Cloudera which sees taking Hadoop to mainstream enterprise customers as its raison d’etre;

“Cloudera can help you install, configure and run Hadoop for large-scale data processing and analysis. Get Cloudera’s Distribution for Hadoop and start working with Big Data today.“

Update: Amazon’s Jeff Barr provides a lot more detail in a post to the AWS Blog.

Related articles by Zemanta

• Learn to develop software for Hadoop (makezine.com)
• Amazon Blows Away Fourth Quarter Earnings Expectations (businessweek.com)
• Grand Challenges in Database Self-Management (perspectives.mvdirona.com)
• Cloud platforms of the future: Hadoop and Eucalyptus (news.cnet.com)
• Cloudera aims to take Hadoop commercial (vnunet.com)
• The Meta Cloud – Flying data centers enter fourth dimension (theregister.co.uk)
• Cloudera floats commercial Hadoop distro (theregister.co.uk)
• Free Cool Tools for Educators (johnmwillis.com)
• Hadoop-Focussed Startup, Cloudera Raises $5 Million (gigaom.com)
• Hadoop, a Free Software Program, Finds Uses Beyond Search (nytimes.com)
• Cloudera launched to offer commercialized Hadoop (insidehpc.com)
• Cloudera (battellemedia.com)
• LinkLog: Streaming Data, Distributed Execution Engines (dorai.wordpress.com)

This Thursday evening, for example, I’ll be down to speak on a panel at the latest mashup* event; the Realtime Social Web.

Given the realities of evening train travel I can’t get home after the event, so have lined up meetings for Friday instead.

And then on Monday it’s back on the train once more, to attend the Powered By Cloud event that I wrote about earlier this month.

I intend to catch up with various friends and acquaintances at both events, and if you’re going to be around do say Hi…

Related articles by Zemanta

• Realtime Social Web – its here now – Event in London 29th Jan

In my recent cull of subscriptions to print media, BusinessWeek had no difficulty whatsoever in avoiding the chop. It consistently offers a useful and timely perspective on events in the world around me, and (subjectively) seems to intelligently consider the tech perspective on things more often than some of its competitors.

Last September, the BusinessWeek.com site rolled out the beta of a new service; Business Exchange. Closely linked to stories in the magazine and features on BusinessWeek.com, the Business Exchange is a fledgling social network within which members can access background material on stories, submit additional resources of their own, and comment on the content they find. Business Exchange links frequently feature prominently at the end of articles in the magazine, but I’m quite surprised at how rarely I find myself explicitly directed back into the magazine from the site. For a quick introduction, see this video of a recent presentation by BusinessWeek Executive Editor, BusinessWeek.com Editor in Chief (and active twitterer), John Byrne.

According to BusinessWeek’s Director of User Participation, Ron Cassalotti, over 1,000 topics have been approved on the site since it opened four months ago. One of those (the Semantic Web, of course) was started by me, but I also consume and contribute content across a range of other topics relevant to my business interests. Much of this activity is relatively passive, but the site also offers the ability to grow a network of like-minded fellow members, to flag items of interest, and to comment on content shared by others.

User profile and topic pages are visible to non-members, and also rank highly in Google; as Ron demonstrated by showing me how much higher my Business Exchange profile page ranks than my far older LinkedIn profile.

Topics are suggested by BusinessWeek staff and by members of the site, and I get the impression that topics tend to be approved if the topic is in-scope (for BusinessWeek readers) and actively discussed out on the open Web. Ron tells me that there is no formal taxonomy for topics, which certainly makes it more straightforward for his team to adapt to evolving member interests and the shifting nature of the News. The lack of a formal taxonomy raises issues of its own, of course. I, for example, followed both ‘Cloud Computing‘ and ‘Cloud Computing Research.’ The former was proposed by Business Exchange member Ralph Perry, and the latter by BusinessWeek Senior Writer Stephen Baker in gathering background content to inform articles in the magazine. For a while I simply cross-posted content to both, but the numbers would suggest that Stephen’s topic is ‘winning,’ and attracting the eyeballs. Presumably at some point a back-end process (or one of Ron’s team) will make a decision to simply merge the two topics?

The network – and its features – are clearly still evolving, and there’s a way to go. I do find myself on the site most days, though, exhibiting web site visiting behaviour that I thought I’d left behind years ago in favour of my RSS reader.

And today? I find that I’m the Business Exchange’s Featured User, there for all the world to see… 

Thanks to Ron and the team, and if you keep up the good work I’ll keep coming back.

My next step, of course, is to move from being Featured User on the Business Exchange to getting my work printed in BusinessWeek itself. Then my social network-sceptical mum really will be impressed!

Related articles by Zemanta

• Inside BusinessWeek’s Social Media Idea: Business Exchange
• Do Social Network Profiles Help Your Business?

As well as linking to ‘usual suspects’ in the Linked Data space such as the CIA Factbook, GeoNames, DBpedia, Musicbrainz and the like, Thomson Reuters is taking the additional step of providing access to parts of their information empire. Already recognised as one of the premiere sources for financial and company information on the global stage, the company is choosing to embed Thomson Reuters services such as stock tickers, information on corporate Boards, etc.

In a world in which everyone can participate, and in which citizen journalism, crowd sourcing and the rest are leading to powerful and wide-ranging disruptions to traditional media, Thomson Reuters is betting that authoritative, timely and branded data has value, and that opportunities exist to build that value by giving some previously expensive information away for free in order to jump-start a range of new and lucrative services.

Freebase employees once talked about a desire to become the canonical source of certain facts on the web; the go-to place for a class of information. Might Thomson Reuters have similar ambitions in the business information market, and might the undeniable (and free) value of the Calais web services be the hook that encourages a whole swathe of developers building applications for this market to perpetuate – and strengthen – the company’s reach?

This isn’t cynical, this isn’t devious, this isn’t ‘wrong,’ and there is absolutely no suggestion that the currently free services will ever incur charges for use. Instead, it’s an interesting example of a titan of Old Media thinking creatively about ways in which it can continue to have value in a changing world.

Casa Battló © Larry Miller 2003, and shared on Flickr under Creative Commons License.

Related articles by Zemanta

• Updated Profile: Calais
• Calais Release 4 and the Linking Data cloud…
• Cross-Pollinating DBpedia and Freebase
• Freebase Officially Linked Data with Release of RDF Service

“Now that is actually different than what many Cloud aficionados are doing. They, I would argue, are still focusing on how infrastructure and software will be the difference in the Cloud. I don’t feel that way. The real difference that the cloud will bring about will be in how people interact with the services they care about.”

Using his admiration for the iPhone as the hook (yes, I like mine too), Daryl goes on to argue that;

“Make no mistake. The cloud is about services – not about infrastructure or software. And, what people do with those services will be the most telling bits of reality surrounding this emerging phenomenon called Cloud computing.”

The infrastructure and the software are, of course, vitally important components in realising the Cloud’s potential, but Daryl clearly has a point when he reminds us that we’re all doing this for a reason. A significant proportion of those getting excited about the Cloud today are ‘just’ getting excited about the technology. They’re getting excited about speed, and size, and APIs, and technological disruption.

I’m certainly interested in the technology, but I get excited by the things that become possible when it is put to work; when large sets of resources are put in the hands of a large and interconnected network of people.

That’s why I see so much opportunity in the convergence between the technological, social, economic and strategic threads so loosely labelled ‘Web 2.0,’ ‘Cloud Computing,’ ‘Semantic Web,’ and ‘Linked Data.’ Alone, each is technically interesting (and, possibly, even exciting.) Together, they move us to a whole different level. And that is what this site will increasingly be about.

It sounds as if Daryl may share at least some of those sentiments, and I look forward to the journey.

Related articles by Zemanta

• Cloud Computing: Weapons of mass disruption
• Vendors air the cloud’s pros and cons
• Gartner picks tech top 10 for 2009
• Access your files in the cloud with the iPhone
• Cloud Computing: It’s the destination, not the journey that is important
• Semantic Web and the Cloud have much in common