Image via Wikipedia

Last week, open standards body OASIS unveiled yet another shiny new standards effort. The OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) Technical Committee hopes to make it “easier to deploy cloud applications without vendor lock-in,” and to support moving from one cloud to another. The usual suspects — the likes of IBM, CA, and Cisco — are on board. The usual holdouts — Google and Amazon, of course — are not. So what is TOSCA trying to achieve? How does it fit alongside all the dead, dying, or ponderously deliberating cloud standardisation efforts that have gone before? And without the giants of the cloud, is there really any point bothering?

As I’ve probably mentioned before, involvement in various national and international standardisation efforts played a big part in my early career. I went to the working group meetings in odd (but often beautiful) locations. I participated in the conference calls. I engaged on the mailing lists. I drafted and edited and reviewed the documents. I completely buy into the idea that there is a place for foundational standards, developed through consensus-building and maintained for the long haul by organisations that stand apart from the vested interests and their competing agendas.

I also believe that there’s a time and a place for these standardisation efforts. Do it too soon, and we end up ossifying something that needs to be in a state of flux. When you don’t know what the best way to prepare a meal is, it’s too soon to print the recipe book. We need to try different approaches, and we need to be able to throw away the attempts that didn’t work out. More worryingly, standardisation efforts can be used for political ends. They can be little more than a rod with which to beat the (usually dominant) competition. At best a distraction, or a talking shop for those unwilling or unable to just get on and do something. At worst, one amongst a toolchest of dirty tricks in a broader war for hearts, minds, and — ultimately — wallets.

The cloud market is a fascinating place. There are leaders and there are followers. There is innovation, and there is competition. There is agreement, and there is debate. For all the rhetoric, and all the posturing, we really don’t yet know the right answer to many of the cloud’s questions.

Maybe TOSCA and the Open Data Center Alliance and IEEE and the rest are — still — too early, and should be content to let the market thrash out a few more of these issues before anyone tries to write anything down? And when it is time to write some stuff down, let’s make sure we focus on specific, finite, tangible, atomic tasks rather than “the cloud.” As Dave Roberts commented in regard to TOSCA’s scope;

“That goal is so large, that I think it’s probably unbounded. When problems get unbounded, the best you can ever hope to achieve is to solve a large enough subset of the problem that the solution is still interesting. If you can’t achieve that, people ignore the solution because it fundamentally doesn’t help them. There is always an ‘interesting’ part of the problem space that they have to solve a different way, and that undercuts the use of the partial ‘solution.’”

And as for Tosca? Things didn’t end well for her, did they? Might TOSCA’s fate, too, be sealed?

Related articles

• Tech giants back standard for cloud portability – InfoWorld (infoworld.com)
• Proposed spec aims to nix cloud lock-in (gigaom.com)
• On TOSCA and Cloud Standards. MyPOV (diversity.net.nz)

I’ve been a fan of Om Malik‘s boutique analyst site, GigaOM Pro, pretty much from the outset, and happily renew my subscription each year. The site covers a wide range of industry topics, and those Quarterly Wrap-ups are worth the fee all by themselves.

I’ve written a few reports for them in the past, but was delighted when Mike Wolf got in touch to see if I fancied trying my hand at curation on their Infrastructure/Cloud channel.

So next week (from 31 January) I’m going to be gathering and commenting upon links from around the web, writing a daily ‘Today in Infrastructure,’ and finishing off with a Weekly Update. If you’re not (yet!) a subscriber, why not sign up for a free seven day trial and join me for the start of my little adventure?

If there’s something you think I should be covering, do let me know.

Related articles

• Free GigaOM Pro Webinar: The Scalable Cloud (gigaom.com)
• GigaOm raises $2.5M, claims 10,000 Pro subscribers (venturebeat.com)
• The Cloud has a place, even inside heavily regulated industries (cloudofdata.com)
• Talking Scalable Clouds with GigaOM Pro and Limelight Networks (cloudofdata.com)
• GigaOM Structure 2011 (datacenterknowledge.com)
• Join GigaOM at Big Data on March 23 in New York City (gigaom.com)

The OpenStack activity was unveiled by Rackspace, NASA, and their partners back in July, and is on track to deliver functional initial releases in the next few weeks. We discuss the relationship between OpenStack’s deliverables and earlier developments from Rackspace and NASA’s Nebula project, and begin to explore the implications of an Open Source Cloud Computing stack for the wider industry.

This podcast was recorded on Friday 10 September, 2010.

During our conversation we referred to the following resources;

• Amazon
• Amazon EC2
• Amazon S3
• Apache license
• Citrix
• Cloud.com
• Dell
• GoGrid
• HP (and my podcast with Russ Daniels)
• IBM (and my podcasts with Kristof Kloeckner and Ric Telford)
• KVM
• Linux
• Microsoft (and my podcast with Amitabh Srivastava)
• mySQL
• NASA
• NASA Nebula (and my article for GigaOM Pro)
• OpenStack
• OpenStack code launchpad
• OpenStack design conference, November 2010
• Oracle
• Rackspace (and my podcast with Lew Moorman)
• Rackspace CloudFiles
• VirtualBox
• Werner Vogels, Amazon CTO, discussing ‘false Clouds’ at GigaOM’s Structure conference
• XenServer

Related articles by Zemanta

• OpenStack sets some Texas-sized deadlines (zdnet.com)
• OpenStack: Rackspace and NASA Nebula Join Forces for Open Cloud Ecosystem (readwriteweb.com)
• NASA gives OpenStack instant credibility (zdnet.com)
• Impact of OpenStack Project Goes Beyond the Cloud Industry Leaders (readwriteweb.com)
• Space IT, the final frontier (radar.oreilly.com)
• What’s next for OpenStack’s cloud efforts (news.cnet.com)
• Rackspace Announces Open Source Cloud Platform (informationweek.com)

When I was a child, I wanted to be a spaceman. No great surprise there, perhaps, and also no great surprise that I — like so many others — never got to achieve that dream.

Still, when the opportunity presented itself to write a space-y piece as my latest contribution over on GigaOM Pro, I jumped at it.

It was fascinating to hear about the very different efforts at the European Space Agency and NASA to harness Cloud Computing… and it brought back some of that childhood excitement in the process.

Thanks to William O’Mullane at ESA and Chris Kemp at NASA Ames for their time and insight.

And if a slot opens up on the next non-robot mission, you know where to reach me…

Related articles by Zemanta

• NASA’s Nebula: The Cloud in a Container (datacenterknowledge.com)
• To Space and Beyond: The Rise of Research-driven Cloud Computing (gigaom.com)
• Cassini Shows Saturnian Roller Derby, Strange Weather (spacefellowship.com)
• Ames expects Obama budget will mean growth (insidehpc.com)

I shared some of my views on this site soon after the event, but think you’ll like the range of opinions and insights from regular Gang members, participants in our audience, and conference organiser Tony Shaw. Have a listen, and see what you think.

As well as generating our regular audio podcast, the event organisers were kind enough to also video the session, and publish the video on their site. I’ve also embedded it, below.

In fact, there’s a nice set of resources from the conference taking shape over on the Semantic Universe site; take a look and see video from the Linked Data Panel (another of the sessions I moderated), the keynote sessions and more.

Related articles by Zemanta

• Tom Gruber talks about Siri, the Virtual Personal Assistant (cloudofdata.com)
• Don’t Miss: Semantic Web Gang, Web 3.0 Conference, Semantic Technology Conference (semanticsincorporated.com)
• Bing is not alone; similar techniques alive and well in existing vertical search (blogs.zdnet.com)

Image via Wikipedia

I mentioned the Semantic Web Gang podcast last week, in the context of our upcoming Live appearance at the Semantic Technology Conference in San Jose next month.

This month’s show was recorded yesterday, and is now available. During the conversation, Gang members dig into the two hot stories of the moment; the launch of Wolfram Alpha and Google’s apparent embracing of semantics with their ‘Rich Snippets.’

Have a listen, and see what you think.

The Semantic Web Gang podcasts are sponsored by Talis. Show notes are available.

Related articles by Zemanta

• The Semantic Web Gang – LIVE in San Jose (cloudofdata.com)

Amid mounting international concern that the guidance lasers aboard Jeff Bezos‘ new Floating Amazon Cloud Environment would interfere with Rudolph‘s sense of direction, sources close to the Amazon Web Services team tell me that they’ve been forced to alter priorities and switch attention to an early release of the next product on their roadmap.

Today sees the release of Amazon‘s latest web service; the Hadoop-powered Elastic MapReduce;

“Using Amazon Elastic MapReduce, you can instantly provision as much or as little capacity as you like to perform data-intensive tasks for applications such as web indexing, data mining, log file analysis, machine learning, financial analysis, scientific simulation, and bioinformatics research. Amazon Elastic MapReduce lets you focus on crunching or analyzing your data without having to worry about time-consuming set-up, management or tuning of Hadoop clusters or the compute capacity upon which they sit.”

The company’s press release quotes VP for Product Management & Developer Relations, Adam Selipsky, who notes;

“Some researchers and developers already run Hadoop on Amazon EC2, and many of them have asked for even simpler tools for large-scale data analysis. Amazon Elastic MapReduce makes crunching in the cloud much easier as it dramatically reduces the time, effort, complexity and cost of performing data-intensive tasks.”

MapReduce was brought to prominence by Google, and is one of the principal techniques at that company’s disposal in enabling them to break massive data sets into manageable chunks suitable for cost-effective processing on the commodity hardware for which they are known. The abstract for a Google research paper on the topic outlines the value proposition reasonably succinctly;

“MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key. Many real world tasks are expressible in this model, as shown in the paper.

Programs written in this functional style are automatically parallelized and executed on a large cluster of commodity machines. The run-time system takes care of the details of partitioning the input data, scheduling the program’s execution across a set of machines, handling machine failures, and managing the required inter-machine communication. This allows programmers without any experience with parallel and distributed systems to easily utilize the resources of a large distributed system.

Our implementation of MapReduce runs on a large cluster of commodity machines and is highly scalable: a typical MapReduce computation processes many terabytes of data on thousands of machines. Programmers find the system easy to use: hundreds of MapReduce programs have been implemented and upwards of one thousand MapReduce jobs are executed on Google’s clusters every day.“

Hadoop is a Yahoo!-nurtured Open Source equivalent to Google’s MapReduce, managed as a project of the Apache Software Foundation, and reputedly scalable to handle many petabytes of data distributed across thousands of CPUs.

As Adam noted in the press release, customers (such as the New York Times and Netflix) are already using Hadoop on Amazon’s Web Services. Today’s announcement makes it easier to cost-effectively and transparently commission (and decommission) the required compute resources. This is the ‘elasticity’ referred to in the new service’s name, and is an increasingly important aspect of the current generation of Cloud-based compute services; much of the economic value proposition lies in only using (and therefore paying for) the resources you actually need to complete a task. If demand increases, the number of (virtual) machines available should rapidly increase to cope, and they should shut back down just as rapidly when the demand passes;

“Amazon Elastic MapReduce enables you to use as many or as few compute instances running Hadoop as you want. You can commission one, hundreds, or even thousands of instances to process gigabytes, terabytes, or even petabytes of data. And, you can run as many job flows concurrently as you wish. You can instantly spin up large Hadoop job flows which will start processing within minutes, not hours or days. When your job flow completes, unless you specify otherwise, the service automatically tears down your instances.“

Elastic MapReduce is currently available only for data centres in Amazon’s US region (so non-US customers can use the service; they just have to be able/willing to transfer the data beyond their borders), and is priced in addition to existing EC2 instances with Elastic MapReduce on a $US0.10 per hour ‘small’ instance costing a further $US0.015 per hour (yes, 1 and a half cents per hour) and on a $US0.80 per hour ‘extra large’ instance costing a further $US0.12 per hour.

Elastic MapReduce is another nice example of slow, incremental improvement to Amazon’s core Web Services offer.

It remains to be seen, as developers get down to using it for real, whether it’s pitched as a low-end disruptor that simply rounds out another piece of the emerging AWS whole, or if it’s a viable competitor in its own right to the recently announced Cloudera which sees taking Hadoop to mainstream enterprise customers as its raison d’etre;

“Cloudera can help you install, configure and run Hadoop for large-scale data processing and analysis. Get Cloudera’s Distribution for Hadoop and start working with Big Data today.“

Update: Amazon’s Jeff Barr provides a lot more detail in a post to the AWS Blog.

Related articles by Zemanta

• Learn to develop software for Hadoop (makezine.com)
• Amazon Blows Away Fourth Quarter Earnings Expectations (businessweek.com)
• Grand Challenges in Database Self-Management (perspectives.mvdirona.com)
• Cloud platforms of the future: Hadoop and Eucalyptus (news.cnet.com)
• Cloudera aims to take Hadoop commercial (vnunet.com)
• The Meta Cloud – Flying data centers enter fourth dimension (theregister.co.uk)
• Cloudera floats commercial Hadoop distro (theregister.co.uk)
• Free Cool Tools for Educators (johnmwillis.com)
• Hadoop-Focussed Startup, Cloudera Raises $5 Million (gigaom.com)
• Hadoop, a Free Software Program, Finds Uses Beyond Search (nytimes.com)
• Cloudera launched to offer commercialized Hadoop (insidehpc.com)
• Cloudera (battellemedia.com)
• LinkLog: Streaming Data, Distributed Execution Engines (dorai.wordpress.com)

The company was founded in 2003, and is currently profitable with some thirty employees, no debt and no external financing.

According to Malviya, the team set out to create a multi-tenant Platform optimised for the easy creation of web-based applications, but their first public offering was a SaaS CRM application devoted to the media industry. That application is still running today, and is used by more than 150 enterprise customers.

The Platform followed in September 2007, exclusively as a hosted offering from LongJump’s partner data centres, and Malviya perceives it as in direct competition to Salesforce.com’s Force.com. With extensive support for common protocols and communications specifications such as REST, SOAP, Java and XML, and a common security framework throughout, Malviya argues that an application Platform such as the one his company offers is important in overcoming what he describes as the

“Enterprise tendency to develop one-off apps… and then expect IT to support them.”

He argues that business users are able to take a degree of control over their own application needs, whilst reassuring the CIO and IT Team that everything is running on top of a single set of secure infrastructure over which they have control.

Today, that same Platform is being made available on an annual subscription basis for local installation behind the firewall, as well as being offered to a range of third party hosting companies whom Malviya hopes will brand and resell the software to their own customers. According to Malviya, two (unnamed) service providers have already agreed to resell the Platform, and details will no doubt be forthcoming as they get up to speed.

As the LongJump corporate site notes,

“LongJump BAP for Enterprises addresses a major gap in next generation application platforms designed solely for the public web. Information that is sensitive or subject to regulatory issues is relegated to sitting on a custom application built internally, adding to a level of complexity and management that neither scales nor adapts well with changing requirements. Because LongJump BAP for Enterprises sits inside your own datacenter, its applications and platform are ideal for organizations tasked with managing [sensitive] information”

And, crucially,

“In addition, applications can be packaged for migration from one LongJump deployed instance to another. For example, development can take place in a cloud deployment and be deployed in production in a corporation’s data center.”

This will, of course, be crucial in allowing flexible movement between different environments as requirements, capacity and resources change.

Related articles by Zemanta

• Marketers Beware: Even Dilbert Says Everything is a Platform (marklogic.blogspot.com)
• Mashups Quickly Emerging through PaaS (programmableweb.com)
• Developers are bullish on PaaS (infoworld.com)
• PaaS risks (accmanpro.com)
• PaaS, Trusting Beyond Its Initial Hype (cloudave.com)
• Salesforce.com: Pondering the next 10 years (news.cnet.com)
• Understanding SaaS business models in conversation with Adam Gross of Salesforce.com (cloudofdata.com)

Salesforce.com

More recently the company has begun to diversify from its heritage as the provider of an on-demand CRM application, seeking to nurture an ecosystem of add-ons and enhancements through the AppExchange and offering third party developers access to the underlying Force.com Platform.

In an effort to understand the company’s views on the evolving SaaS and PaaS markets I recently spoke with VP for Developer Marketing, Adam Gross, and the result has just been released as a podcast.

Show notes available on Talis‘ Nodalities blog

Delivery of both application and platform could create significant tensions, as business decisions made to advance the application potentially cannibalise revenue from the platform ecosystem, and vice versa. We discuss some of these issues during the conversation, with Adam going so far as to suggest that ‘nothing would stop’ a third party using Force.com to build an application that competed directly with Salesforce itself.

Have a listen, and see what you think.

Image of Adam Gross © Salesforce.com

Related articles by Zemanta

• How to evaluate software-as-a-service for your business (news.cnet.com)
• Salesforce.com outage hits thousands of businesses (news.cnet.com)
• Salesforce hits its stride (money.cnn.com)
• Enterprise Software is Not Dead Yet (ventureblog.com)
• Developers are bullish on PaaS (infoworld.com)
• Planning necessary for corporate SaaS (macworld.com)
• Mashups Quickly Emerging through PaaS (programmableweb.com)
• PaaS risks (accmanpro.com)
• Salesforce.com builds another bridge to Google’s cloud (venturebeat.com)
• Salesforce links Force.com to Google App Engine (infoworld.com)
• Salesforce.com squeezes $1B from the cloud (news.cnet.com)
• Q&A;: 10 questions with Salesforce’s Marc Benioff (news.cnet.com)
• Salesforce.com Preaches Computing Power for Rent (nytimes.com)
• Salesforce Adds Twitter, Teases Rivals (blogs.wsj.com)
• Apprenda CEO Sinclair Schuller talks about their SaaSGrid Platform (blogs.talis.com)

In conversation with RightScale CEO Michael Crandell yesterday it would appear that he, at least, sees little need to join this scramble upward.

Show notes for this podcast are available on Talis‘ Nodalities blog.

For those with absolutely no idea what I’m talking about, it’s worth taking a moment to step back and describe this Cloud Computing stack. At its simplest (and this is quite complex and descriptive enough for almost every real-world conversation, thank you very much), the stack can simply be characterised as Applications (or Software), which sit atop Platforms, which sit atop Infrastructure. I have lifted four slides from a recent presentation, to graphically illustrate the stack and to show some of the players found at each layer.

Each layer of the stack is extremely dependent upon the one below, and there is a tendency for the lower levels to be perceived as more of a commodity or utility. As such, pricing tends to be highly competitive and margins on individual transactions are (often) vanishingly small, requiring successful players to operate at scale in order to effectively generate revenue.

Assuming these presumptions to be true, it’s easy to see why those operating at and near the Infrastructure layer might wish to progress upward to a space where there is less direct competition and far more opportunity to charge premium prices.

As Michael and I discussed in our podcast ‘basic’ Infrastructure providers such as Amazon, GoGrid, Rackspace and others are increasingly seeking to enrich their own offerings, adding dashboard and management functionality that was previously the preserve of value-adding third parties such as Michael’s company.

Faced with pricing pressure from direct competitors and potential feature creep from below, it’s certainly easy to believe the truism with which this post began.

Have a listen, though, and hear a rather different view of the space from Michael… who seems perfectly happy where he is in the stack.

Related articles by Zemanta

• RightScale Gets $13M To Manage the Cloud (gigaom.com)
• Cloud management startup RightScale raises another $13M (venturebeat.com)
• Executive Briefing on Cloud Computing (aws.typepad.com)
• Cloud computing: Don’t get caught without an exit strategy (infoworld.com)
• Without APIs, there is no cloud computing (news.cnet.com)
• What is Cloud Computing … (gardeviance.org)
• The Cloud is the Next Step in Enterprise Computing (byronmiller.typepad.com)
• Amazon Crushes Ecosystem – Launches AWS Management Console (elasticvapor.com)