Image © Mission Bay Conference Center

Last month, RightScale’s State of the Cloud report got me thinking about the rise of multi-cloud solutions. Next month, I’ll be moderating a Mapping Session at GigaOM’s Structure event to work out how, where, when, why and if this trend is going to prove significant.

Hybrid clouds, in which one public cloud and one private cloud are used together, are becoming increasingly common solutions to a range of business challenges. RightScale’s figures suggest growing interest in something more complex and, potentially, more interesting; multi-cloud. In a multi-cloud arrangement, customers build solutions combining one or more public clouds with one or more private clouds. This has the potential to significantly increase complexity, without necessarily delivering a comparable increase in value.

In my post last month, I suggested that many of these multi-cloud deployments were essentially accidental. A quick email exchange with RightScale shows that their survey respondents would appear to disagree; multi-cloud, for them, is a conscious business decision.

I’m intrigued, and so were GigaOM. So we’re putting on a Mapping Session at next month’s Structure conference to explore the issue further. I’ll be joined at the front of the room by fellow GigaOM Pro Analysts Ben Kepes and David Linthicum, as well as GigaOM Research Director Jo Maitland.

A GigaOM Mapping Session is more like a workshop than a regular panel. Although there are analysts at the front of the room, they’re really there to stimulate and guide a conversation with every single person in the room. We don’t have all the answers. We’re there to explore the topic, and to work out what — if anything — it might mean. The perspectives of customers, practitioners, suppliers and investors are an integral part of the process. When it works well, everyone comes away with a broader perspective than when they entered the room. Ideas are born, perceptions are sharpened, the germs of deals are done, and directions for future GigaOM research are painted. Mapping Sessions are free for registered Structure attendees, but separate registration is required. Numbers are capped, to ensure plenty of opportunity for discussion. If you’ve got perspectives to share, please do register to join us on 20 June in San Francisco.

Multi-cloud clearly throws up a host of intriguing issues for cloud vendors, their customers, and the ecosystem of cloud management-type providers (like survey author RightScale). What, if anything, do cloud vendors need to do in order to encourage, support, or hinder multi-cloud adoption? What’s the value proposition behind multi-cloud for customers, and what do the providers of cloud management services need to do in order to capitalise upon an emerging trend? Is this a long-term trend, or a short-lived opportunity?

Many of those stakeholders will be in the room, speaking, listening, and interacting. Do join us, and them.

From time to time, I write about Open Data. This has nothing to do with that. The Open Data Center Alliance is interested in data centres, not data. The Alliance was established back in 2010 with Intel driving things forward, and now claims over 300 member organisations, including the likes of BMW, Lockheed Martin, Microsoft, Deutsche Bank and Marriott Hotels.

According to the ODCA website,

we came together to deliver a unified voice for emerging data center and cloud computing requirements. Our mission is to speed the migration to cloud computing by enabling the solution and service ecosystem to address IT requirements with the highest level of interoperability and standards.

Much of the Alliance’s work involves identifying customer requirements and capturing these in a series of usage models. In theory, prospective customers can modify these usage models in defining their own requirements, and suppliers can tap new business by aligning their offerings to the models. I’ve not seen much evidence that this is happening at scale yet, but the Alliance site does state that

we anticipate quick industry response to the requirements, initial POCs of solutions beginning in 2012. This could accelerate over $50 billion in cloud service investments and is expected to save $25 billion through IT operational efficiency due to cloud adoption.

One of those usage models is concerned with Virtual Machine Interoperability in a Hybrid Cloud Environment (pdf), and last month it was augmented by the release of a Proof of Concept document (pdf) which

outlines testing criteria and procedures for documenting how hypervisor and VM solutions from both ODCA members and non-members interoperated in real-world enterprise cloud scenarios.

Quoted in an April press release to mark publication of the PoC document, ODCA executive director Marvin Wheeler commented that

true interoperability of hypervisor and virtual machine solutions between clouds is critical to advancing enterprise ready cloud implementations. We encourage global IT managers and cloud solution and service providers to join us at the Forecast 2013 VM interoperability panel on June 17 to take part in the collaborative discussions that will help drive the next phase of VM interoperability in the enterprise cloud.

The basic premise is a simple one; a virtual machine started on one hypervisor or class of physical server should run equally well when moved to run on a different hypervisor or physical server. Further, there is a presumption that there is a credible business requirement for this capability. As enterprise users of cloud increasingly find themselves adopting a hybrid approach spanning their own data centre, co-location facilities, hosting sites and diverse public clouds, the likelihood of them needing to run their standard Windows or Linux virtual machines on top of more than one hypervisor certainly increases. It’s less clear, though, that an inability to reliably move a Linux VM from the KVM hypervisor in your own data centre to the Xen hypervisor at a cloud provider is causing significant business pain today; it’s often reasonably straightforward, for example, to simply select a different cloud provider able to support your chosen KVM hypervisor.

But even if a lack of VM interoperability isn’t presenting an insurmountable barrier to business right now, it’s still one more thing to think about when pulling a set of disparate services together. If we can cost-effectively and pragmatically remove or diminish its ability to complicate, then that’s presumably a good thing.

ODCA’s PoC work took the usage requirements the organisation had already defined, and applied them to a specific set of documented tests. As IDG’s Joab Jackson notes in his InfoWorld piece, the results were not great.

Running through all the different possible combinations of hypervisors and OSes, the researchers found that 13 test cases resulted in warnings, and 19 test cases failed entirely. Only in two cases did the VM work flawlessly across two different hypervisors. In both of these cases, a VM created with Xen worked without troubles on a Microsoft hyper-V environment — in one case running Ubuntu and in the other case running Windows Server.

The researchers do note that they set the bar for success pretty high, and that several of the ‘warning’ states would still result in a functional VM. It does appear clear, though, that customers with a real need for multi-VM interoperability face significant challenges in efficiently managing workloads across different virtualisation infrastructures.

And that brings us to the panel at Forecast, which can hopefully help quantify the true scale of the problem… and indicate some of the ways in which vendors are working to fix the broken bits. In terms of the panel discussion itself, I look forward to delving into at least the following:

• how big a problem is the current state of VM interoperability?
• do the ODCA VM Interoperability use cases prioritise the right things?
• what do customers really need, and are they getting it?
• how are vendors really responding to requirements such as those defined by ODCA?

If you have opinions to share, perspectives to offer, facts with which to illuminate, or questions to ask, I do hope you’ll join us.

ODCA has given me a couple of free passes to Forecast. If you want to come along and we know one another, then please do get in touch.

Note: ODCA is covering my flight to San Francisco, and accommodation during the event.

Image of event venue, San Francisco’s Westin St. Francis, by Flickr user ‘prayitno‘

Related articles

• Open Data Center Alliance Tackles Cloud Lock-In (datacenterknowledge.com)
• Virtual machines inching toward portability (infoworld.com)
• 5 Key Essentials of Cloud Workloads Migration (cloudave.com)

But, as we all become increasingly dependent upon data sourced from third parties, aspects of this rather esoteric pastime are beginning to matter to a far broader audience. Products, recommendations, decisions and entire businesses are being constructed on top of data sourced from trusted partners, from new data brokers, from crowdsourced communities, or simply plucked from across the open web. Without an understanding of where that data came from, and how it was collected, interpreted or maintained, all of those products, recommendations, decisions and businesses stand upon very shaky foundations indeed.

Data attribution is increasingly important, but it will be essential to make sure that the rules, tools and norms which emerge are both lightweight and pragmatic. Now is not the time to get heavy-handed and pedantic about where the comma goes.

Former colleague Leigh Dodds recently offered a useful discussion of the rationale behind data attribution. Early on, he describes the related (and, often, sloppily interchangeable) notions of attribution and citation;

It might also be useful to distinguish between:

• Attribution — highlighting the creator/publisher of some data to acknowledge their efforts, conferring reputation
• Citation — providing a link or reference to the data itself, in order to communicate provenance or drive discovery

This distinction is important in some circumstances, but it can also be useful to consider a simpler, more selfish, but ultimately more scalable justification. Attribution (and citation) of data quite simply provides an audit trail, enabling you, your bosses, your investors, or your customers, to know more about the data upon which actions are based.

Creators want credit, consumers want to trust

Much of the serious consideration of attribution comes from a relatively small cadre of data owners or creators, who (understandably, perhaps) want credit for their hard work. Perhaps they want to prove use in order to secure future funding or advancement, or perhaps they simply want to track where their data ends up. Through a series of licenses, contracts and Terms & Conditions statements, these creators have done much in codifying the ways that data should be referred to. Leigh discusses some of the licensing terms in his post, but it’s probably fair to say that none of them have really caught on outside a few rather narrowly scoped groups of co-dependent developers and data providers.

Data owners’ requirements for credit run the gamut, from loosely phrased requests for a link back to their website all the way past this rather excessive example quoted by Leigh to end up as lengthy tomes of draconian legalese;

The attribution must be no smaller than 70% of the size of the largest bit of information used, or 7px, whichever is larger. If you are making the information available via your own API you need to make sure your users comply with all these conditions.

All too often, that sort of self-defeating prescription is enough to send prospective users back to Google, in search of a less demanding alternative.

For consumers of data, or for those wondering where the data behind a product or decision came from, things are rather simpler. On the whole, those on this side of the divide are simply looking for a pointer which enables them to learn more. Is my company’s multi-million dollar change of direction based upon detailed data from stable Governments, credible banks and respected analysts, or did the person responsible use some numbers the found on their friend’s blog?

Carefully crafted rules regarding attribution’s wording, placement, colour, size and typeface are an irrelevance, probably deserving to be ignored or ridiculed.

Far better, and far more likely of success, to simply encourage users and re-users of data to sensibly point back (however they like) to their principal sources.

Remembering all the ancestors is a bit daft

Data set A is modified and added to in order to create data set A1. Data set A1 is modified and added to in order to create data set A2. Data set B is modified and added to in order to create data set B1. Data set B1 is modified and added to in order to create data set B2. Data Set C modifies and extends data sets A2 and B2. It seems reasonable to acknowledge the contribution made to C by A2 and B2, but some would argue (loudly) that A, B, A1 and B1 also need to be acknowledged in C. This is one aspect of ‘attribution stacking’, and attribution stacking is, quite simply, stupid.

If I am the creator of data set C, I am selecting A2 and B2 because they are the right data sets for my purpose. That selection will be based upon a range of criteria, including the scope and coverage of the data. The selection will also be based upon my impression of the brands responsible for A2 and B2, and that impression (implicitly or explicitly) will include some awareness of the processes they use to select, validate and manage the data they use. It’s for them to carefully select, validate and provide attribution for A1 and B1, not for me. And it’s A1 and B1′s job to do the same for A and B, not me.

Things get even worse in some open source data projects, where all the individual contributors expect to be acknowledged. Inside the project (and on its website, etc), that’s fine and sensible. Outside, though? It’s ridiculous. So if data set A were created by individuals Aa, Ab, Ac, Ad and all their friends right up to Az, under some licenses there would be an expectation that every single one of those individuals be acknowledged by name in any mention of data sets A, A1, A2 or C. A massive administrative burden for any downstream users of the data set, and of no real benefit to anyone whatsoever. This desire for glory really does need to be challenged, if it is not to stifle free and fair downstream use and reuse of the data. Within the project building A, it may be vital to know that user Aa is a bit sloppy, or that user Ad has a nasty habit of making the data say what she thinks it should rather than what it actually does. But it is the responsibility of the project behind A to put processes and procedures in place to address these issues, and to ensure that all of its participants receive appropriate credit within the project for their contribution. By the time we reach A1 or A2, though, those internal details no longer matter. A1 chose to use A because those processes exist. After an initial evaluation of those processes and their implementation, A1 can — and should — simply trust them, rather than endlessly second-guessing them.

Tracking and Trust are different

Ultimately, the motivations of data creators and data re-users are very different. The processes and procedures put in place by creators and owners in search of kudos or statistics may actively obstruct the use and reuse that they profess to want. Complex forms of attribution, aligned to heavy-handed enforcement of infringements, do nothing to encourage a far broader community of use to emerge.

By attempting to count — and manage — the small number of uses today, data creators are stifling growth that otherwise is ready to explode. A perfect example of the saying (which may not translate beyond Britain’s shores!) of ‘biting off your nose to spite your face.’ Think about it…

Leigh ends his post with

Attribution should be a social norm that we encourage, strongly, in order to acknowledge the sources of our Open Data.

Other than broadening it from ‘Open Data’ to just ‘data,’ I couldn’t agree more. But let’s keep it lightweight, simple, and pragmatic.

Note: or perhaps this post should have been called “When you stand on a giant’s shoulders, it’s a good idea to say thank you.”

Image of Eduardo Paolozzi‘s sculpture of Sir Isaac Newton by Flickr user ‘monkeywing‘

Related articles

• How Do We Attribute Data? (ldodds.com)

Albert Einstein, you may have heard, was a clever man. He scribbled equations on blackboards, thought big thoughts, and all of that. But, allegedly, he also said

Everything should be made as simple as possible, but not simpler.

These words have resonated with me recently, as I’ve heard pitches from one company after another, all of which are trying to cut through the complexity of data to make it accessible. Their goals appear laudable, but all too often I find myself wondering how simple this stuff can be? If we make it too simple, do we run the risk of unleashing a flood of half-baked ‘analysis,’ undertaken by people who really shouldn’t be allowed near a calculator, let alone a Hadoop cluster? On the other hand there’s a persuasive argument to be made for democratising access to data and tools, freeing organisations from over-reliance upon their new High Priests of Data.

Every data question should not require a data scientist, but maybe we really shouldn’t be making it too easy for people to tackle the hard questions without support from someone who knows what they’re doing.

How simple, then, is too simple? And can we use data in a different way; in order to offer enough simplicity, smartly?

One company which has tried to do that in an interesting way is Datahero, and I spoke with co-founder Chris Neumann this week to learn some more.

There’s been some good coverage of the company over the past few weeks, and if you’ve not come across them before then it’s worth taking a skim through the Related articles at the end of this piece. I won’t bother repeating those pieces here.

The thing that interested me — other than Chris’ obvious passion and enthusiasm for his subject — was the way in which Datahero plans to use a mix of data analysis, user experience design and machine learning in order to guide the user toward analyses and visualisations that are likely to be of use to them. Chris is quick to stress that the company isn’t looking too closely at the data values its customers upload. Instead, the system studies the structure of the data (this column contains dates, this column contains place names, etc) and any associated metadata in order to make recommendations for logical ways to visualise the dataset.

As the number of users grows, the roadmap also includes Amazon-style recommendations. If a lot of other people uploading their quarterly sales forecast graph it in a certain way, then it makes sense to recommend that type of graph to a new user who uploads data with a similar structure. The recommendation won’t always be right, but it should go a long way toward minimising the fear of staring at columns and columns of data without a good idea about where to turn first.

Smart recommendations, clever algorithms, and an engaging UI will not — on their own — turn everyone into a data scientist. But nor, really, should they. What they can do, though, is what Neumann described as “enabling the 99%;” those people who have a dump of data from Mailchimp or Google Analytics or Salesforce or Excel, and who don’t really know how to begin sensibly visualising the multitude of columns and rows of numbers.

It’s an intriguing idea, and it will be interesting to see how successfully the system can deliver value to a potentially huge pool of individuals who find themselves data-rich, but skills-poor.

Even if technically successful, of course, the challenge will be persuading those same users to continue paying for the service. If they think they’ll use it often enough to pay for, won’t they end up acquiring enough of the skills to work with their own data anyway? Unless, of course, Datahero manages to grow and add complexity along with its users…

Image of Albert Einstein, ©1947. Image sourced from the Library of Congress’ Prints and Photographs division with identifier cph.3b46036. Shared on Wikipedia, and deemed to be in the Public Domain.

Related articles

• Datahero Turns Data Into Rich Visuals Without The Need For A Data Analyst (techcrunch.com)
• ‘Data vis for the 99 percent’: DataHero launches its free service (venturebeat.com)
• Datahero launches to the public, making genius-level data science simple, automated and free (thenextweb.com)
• Visualization startup Datahero opens its doors and delivers data analysis for the masses [GigaOM] (gigaom.com)

Enigma pulls data from tens of thousands of public data sets, and then offers up an interface that makes it pretty straightforward to trawl through the whole lot in search of the data points that you actually need. As the company’s Marc DaCosta introduced it, a “search and discovery platform for public data.”

At present, everything in Enigma is publicly available data. It’s mostly from the USA right now, and is acquired by a combination of screen scraping/ crawling of .gov sites… and calling government agencies to request that they ship CDs of offline material. DaCosta stresses that the data is all — theoretically — available to any US citizen, but that it’s not really that easy for them to access. Even with the advent of central sites like data.gov, primary data remains spread across a multitude of portals and web pages, stored in a dizzying array of machine readable formats… and (all too often) as PDFs full of tabular data. DaCosta finishes his preamble by stressing the need for

infrastructure to acquire, index and search public data.

The team’s Disrupt pitch and the subsequent back and forth with the judges provides a pretty good overview. It is worth 12 minutes of your time to watch over on TechCrunch.

During the Q&A, Freestyle Capital’s Dave Samuel raised the same concern I did during my meeting with the Enigma team. The set of possibilities with public data is extremely large, but still finite. Far more so than Google, there are real limitations to the sorts of questions that it makes sense to ask. Presented with a search box, users struggle to understand what’s possible and what’s sensible. Simple tricks would go a long way toward helping here, such as highlighting today’s most popular queries, or providing sets of sample queries as Wolfram Alpha does. Users need to learn how to work with aggregations of data such as this, and the onus is on sites like Enigma to help grow their potential market through lightweight and accessible education.

In many ways, though, I think the biggest opportunity for Enigma doesn’t lie in their website at all. The real opportunity lies in their api, and in licensing it to third parties who will construct entirely new Enigma-powered vertical applications or integrate Enigma data into existing software such as an investor’s due diligence systems.

In and of itself, the data Enigma has gathered is interesting, but neither indispensable nor unique. The real value comes from offering easy, reliable, comprehensive and cost-effective means to get this data integrated into existing workflows, and that’s where the api needs to shine. There’s also an opportunity to encourage/enable companies to upload their own data into the platform, letting them combine it with the public data already there. This is, apparently, on the road map. I would imagine that this corporate data will initially only be available to the company that owns it, but there are a whole set of other opportunities around sharing data with supply chain partners, and even making it available to anyone.

Nice (and fast!) as the website is once you understand how it works, I tend to see it far more as a shop window than as a revenue-generating service in its own right. The Enigma team disagrees, seeing subscription-based access to the website as one of their two main products. We shall see who proves more right in due course!

Finally, for now, the company faces a real challenge in scaling. It needs to pull in more data from inside the US, and it needs to broaden its coverage outside the US. Its data acquisition processes, although highly automated, remain pretty labour intensive. And its data teams need to understand the data they’re working with. Some government data is rigorously documented, whereas other data sets are almost unintelligible. Growth, as the Enigma team recognises all too well, requires far more than simply pointing their crawlers at some new web domains.

March and April were busy with travel. May’s pretty quiet on the travel front, with plenty of opportunity to get some proper work done before heading to Brussels at the end of the month. The next trip States-side looks like being to San Francisco in June. I wonder who I’ll meet there… and what they’ll go on to win?

Picture of the Enigma cryptographic machine by Flickr user Chris Brown.

Related articles

• And The Winner Of TechCrunch Disrupt NY 2013 Is… Enigma! (techcrunch.com)
• This TechCrunch Disrupt Winner Could Be The Future Of Search (businessinsider.com)
• Enigma brings the deep, dark world of public data to light (venturebeat.com)
• Enigma Makes Unearthing And Sifting Through Public Data A Breeze (techcrunch.com)
• Beating out tough competition, ‘Google for public data’ Enigma wins Disrupt NY (venturebeat.com)
• Enigma Reinvents Public Data And Wins TechCrunch Disrupt Battlefield (nibletz.com)

Xeround was founded back in 2005, initially delivering scalable data management solutions into the data centres of telcos such as T-Mobile. The company’s work on scaling MySQL and offering it in the cloud brought it to wider attention, and they seemed to maintain momentum by adding additional cloud partners. Last year, the company rolled out a freemium offering to entice a far wider set of potential customers. It would appear, though, that any influx of paying customers was insufficient to meet the cost of this rapid expansion across disparate cloud providers.

In last night’s blog post, Xeround’s Avigail Ofer writes;

Xeround’s leadership forum has recently decided to re-focus the company’s effort. This means we will no longer be able to support our service over public clouds, across all of our currently active data centers.

This presumably means a return towards the company’s roots, offering scalable data management solutions inside enterprise data centres. As a strategic decision it may make sense, although that market is far more crowded than it was in 2005. For now, though, the company website continues to talk almost exclusively about their Database as a Service (DBaaS) offering, hosted in public clouds. Anyone looking to see what else Xeround does today, or where it will be active after 15 May, will get few clues from the site. Current and prospective customers must, inevitably, feel anxious. What else might be consigned to the pyre?

Strategic refocusing, cutting underperforming products, and even pivoting are facts of business life. Well-known consumer services like Flickr and Twitter were the result of pivots that must have seemed painful at the time. In those cases, and others, pundits and commentators bemoaned the death of whatever went before… but few would dispute the success of the new ventures. Customers worry about what they have. I still miss Tungle, and have no idea how I will cope when Google cruelly sacrifices the invaluable Reader. But boards and managers need to worry about where they’re going. Sometimes, maintaining the status quo simply does not make sense. You’re breaking even or — worse — burning cash. You can see a new competitor that’s faster, better, or cheaper than you. Your revenues or usage figures are headed the wrong way. All are signs that something needs to change. And if you’ve got a better idea about how to focus resources, why not go for it?

Successful companies aren’t necessarily the ones with the most brilliant people, or the strongest teams, or the best idea, or the biggest pile of cash. Successful companies are those with people who are good enough, who can work together well enough, on an idea that’s intriguing enough, attracting sufficient cash to get noticed. If all of those pieces fall into place, successful companies then need a good dose of luck, and a great big dollop of smart timing. The market needs to be ready, customers need to be in the mood to buy, and the competitive landscape needs to be tilted in your favour. Sometimes, those pieces don’t all align, and the company needs to shift focus a little and try again.

Trying again is fine. There’s no shame in it. But when you try again people remember things about how you behaved the last time. They won’t hold failure or pivoting or downsizing or refocusing against you. But if they are considering becoming an investor, partner or customer, they’ll remember how you treated investors, partners, and customers last time around.

And that’s why Xeround’s timing for the shutdown bothers me. A week (or two, if you’ve given them cash) is not a lot of time to move off a database that might be baked deep into your business. Customers must be waking up this morning, and scrambling to adapt. Their roadmap is out the window. Their vacation is on hold. Their work to attract the next client abandoned, as they scrabble to keep the database running for the clients they already have. Xeround did that. And it didn’t need to.

Pivot if you want, Xeround, but you’ll have a much better hope of maintaining current and future custom if you treat your users with respect. And a week or two’s notice for the removal of something that may very well be mission critical is not respectful.

I did try to reach Xeround’s PR firm for comment, without success. I shall update this post if I receive anything relevant from them or the company.

Hat tip to Barb Darrow at GigaOM, who was quick to cover the story… and to update her post as more facts emerged.

Image of a burning Viking ship from istockphoto.

Related articles

• Xeround Provides Auto-Scaling & High-Availability for your MySQL Database in the Cloud – The Official Rackspace Blog (rackspace.com)
• Biggest SQL Disasters (i-programmer.info)

Surveys. Along with the loathsome infographic, they have become the default tool of the lazy PR person. I get loads of them, pretty much every day. And, almost without fail, they are a shocking waste of everyone’s time. Three out of the four people we gave some money to said our product was best. I don’t care. And I’m definitely not going to write about it. Every now and then, though, one that’s worth a little attention comes along. RightScale’s latest State of the Cloud, released last week, is one of those.

The sample is relatively small (just 625) and, as Network World‘s Brandon Butler emphasises, they

came from a community to [sic] people that had somehow, some way interacted with RightSacle [sic] in the past — they had registered on the company’s website, dropped a business card in a bowl at a trade show, or are a customer (although RightScale says only 30% of respondents were customers). The firm’s VP of Marketing Kim Weins says respondents to RightScale’s survey admittedly have some ‘affinity’ to cloud computing.

RightScale’s analysis of respondent demographics bears this out, with over half (51%) describing themselves as being from ‘Tech Software, Services, Hardware’ businesses. The broader community of cloud-consumers in manufacturing, financial services, media and the rest is far less well represented.

Survey Demographics. Image © RightScale 2013

The survey digs into a number of areas (see the Related articles, below), but it was the discussion of ‘multi-cloud’ that particularly caught my attention.

Image © RightScale 2013

According to the survey, 77% of respondents from enterprises with 1,000+ employees (see Disclaimer for an important proviso) had some sort of multi-cloud policy. The majority of those (47%) are pursuing some form of on- and off-premise hybrid strategy, whilst 15% are employing multiple private clouds and 15% are employing multiple public clouds. Amongst enterprises with 1,000+ employees that are already using a public cloud solution, the numbers are even higher; 88% are pursuing a multi-cloud policy, with 53% opting for a hybrid approach, 22% for multiple public clouds, and 13% for multiple private clouds. In both cases, it is unclear whether respondents could describe themselves as using multiple public and multiple private clouds.

A hybrid approach, in which some computing is done on-premise (or in a long-term and highly managed co-location or hosting arrangement) and some is done in the public cloud makes an awful lot of sense. Some data may be deemed too expensive or too sensitive to routinely move in and out of the cloud. Some workloads are steady, routine, and highly optimised to run most cost-effectively in a supported, SLA-wrapped and tightly managed dedicated facility. Other workloads demonstrate dramatic peaks and troughs in usage, or require irregular access to resources which are too expensive to maintain — frequently idle — in-house. Even in the most regulated, paranoid or old-fashioned organisations, there are plenty of workloads for which a hybrid solution is ideal.

More interesting than the simple use of a hybrid solution is the suggestion that companies are using more than one public cloud, more than one private cloud, and perhaps more than one of both. There is, perhaps, a clear opportunity here for cloud management solutions (such as RightScale’s, of course), but to really understand the nature of the opportunity we need to understand far more about why the trend we’re seeing is occurring. There’s insufficient data in RightScale’s published results to be sure, but possible explanations include, one, some, or all of the following.

Eggs and Baskets

It’s a conscious decision. Public clouds suffer outages. Private cloud providers get acquired, go bust, or change direction. To continue delivering core services to key customers, the modern enterprise is forced to spread its services across different solutions providers. The gradually increasing credibility of enterprise cloud solutions from HP, IBM, Microsoft and others may diminish this perceived requirement.

The Search for Features

It may be deliberate, but is more likely to be an accidental byproduct of something else. Different public and private cloud providers excel in different areas. They provide support for different virtual machine instances, their apis are better at some actions than others, they offer services in different jurisdictions, and with varying levels of support. The modern enterprise, in meeting a wide range of business requirements, will inevitably find itself selecting different clouds for different reasons. This is likely to continue for some time to come, despite the growing feature completeness of competing offerings.

The Departmental Power Grab

It may be deliberate, but is more likely to be an accidental byproduct of something else. The organisation has no centralised policy regarding the cloud. Different departments and divisions are pursuing their own formal strategies. Their differing requirements lead, almost inevitably, to the selection of different solutions. As the CIO attempts to reassert control, there may be scope for solutions providing a sensible view across these competing services. Alternatively, there may be an explicit move to consolidate, creating opportunities for those able to assist in migrating services from one provider to another.

Shadow IT

It may be deliberate, but is more likely to be an accidental byproduct of something else. The different solutions are being selected by individuals and small teams. They may be selecting those they’ve previously used elsewhere, or they may be employing a wide range of criteria from the frivolous to the robust. Differing requirements and perspectives lead to a wide range of overlapping services being selected. If the CIO attempts to reassert control, there may be scope for solutions providing a sensible view across these competing services. Alternatively, there may be an explicit move to consolidate, creating opportunities for those able to assist in migrating services from one provider to another.

 Play Time

It may be deliberate, but is more likely to be an accidental byproduct of something else. Developers are casting the net wide, experimenting and exploring. The organisation may be littered with accounts for different cloud services that someone tried, explored for a while, and then moved away from. There may well be a large number of intranet-type applications that were built during the exploration phase, deployed to a cohort of internal users, and then simply left running.

Oops

It’s unlikely to have been a conscious business decision. No one knew someone else had signed up for that other cloud. The person who did it left last year, and we’re really not sure if there are some nasty dependencies in our upcoming product…

What do you think? Why are we seeing people using more than one public cloud, or more than one private cloud? Is it a trend that’s likely to continue? Is it an opportunity, or a cause for concern?

I suspect that the vast majority of ‘multi-public’ and ‘multi-private’ responses are not the result of a conscious and explicit decision to use more than one cloud. Instead, they’re simply a byproduct of the industry’s current state of evolution. One cloud, today, is better for some things, and its competitor is better for others. Mainstream enterprise adopters need both those capabilities, and therefore end up having to use two clouds.

Potential adopters of multi-cloud, though, would benefit from a more considered exploration of the issues. There are good reasons to learn and use more than one cloud, but there is significant scope for increased complexity in following such a path. Cloud management providers like Enstratius and RightScale may diminish some of that complexity, but it doesn’t go away altogether.

Multi-cloud should be a conscious decision.

Disclaimer: It is important to note that the RightScale survey does not identify the number of responses upon which analysis is being performed. The total number of survey respondents was 625, but subsequent discussion in their report appears to be based upon subsets of the whole. Figures on enterprise cloud usage, for example, are drawn from “organisations with a hybrid cloud strategy” (p.6). There is no information on the proportion of respondents classed as ‘enterprise,’ nor the proportion of those that may have a cloud strategy. The information that 29% currently run apps in a hybrid cloud may point to 29% of 625, or 29% of 3. Similarly, discussion (p.7) of multi-cloud implementation is based upon enterprise respondents with more than 1,000 employees! Unhelpfully, the descriptive text uses “all respondents,” “all enterprise respondents,” “respondents” and “enterprises” interchangeably. It would be useful if RightScale could provide explicit sample sizes next to each of their graphs.

Image of feet in a bed by Flickr user Nany Mata.

Related articles

• RightScale survey points toward preference for ‘multi-cloud’ deployments (zdnet.com)
• RightScale sees uptick in cloud adoption and multi-cloud use [GigaOM] (gigaom.com)
• Cloud adoption growing, but how big depends on who you ask (networkworld.com)
• RightScale State of the Cloud 2013: A New Industry Survey (rightscale.com)
• With Experience Comes Wisdom in Cloud Computing, Survey Shows (forbes.com)
• Survey Says: Multi-Cloud Usage Growing, DevOps on the Rise (datacenterknowledge.com)

As the Big Data hype machine continues its relentless attempt to gobble everything in its path, new business units and entire new domains buying into the promise find themselves faced with unanticipated data volume and complexity. They see the potential for data-based decision making, but still face (short-term?) challenges in actually managing, analysing or interpreting the data they now collect.

Early iterations of core tools such as Hadoop were raw and unpolished, driving the emergence of a niche group of developers and data analysts with the specialist skills to cope. Those tools become easier to use with each new release, which goes some way toward countering panicked claims of a massively yawning skills gap.

There is, of course, still a need for people with specialist knowledge, hard-won skills, and painfully gained experience. But you no longer (if you ever really did) need to install a Hadoop cluster with your bare hands and juggle complex statistical formulae in your head to benefit from the growing prevalence of data in all aspects of business.

At the ‘softer’ end of the market, specifically, there has been an explosion of new startups rushing to offer tools that make it easier to create visualisations and dashboards to deliver some value from the data whilst hiding its complexity. Some established players in the visualisation and business analytics market, such as IPO-prepping Tableau and IPO-considering Rosslyn Analytics, are hoping to press home their lead. Elsewhere, visualisation and business intelligence pros eye the influx of new users with horror, reprising the wailing of typesetters and designers when desktop publishing tools like Ventura and PageMaker first appeared; these amateurs, they protest, cannot do it properly. They do not understand. They — shock, horror — might mislead people.

We’ve been here before. A $50bn market opportunity will tend to see smart young things — and charlatans — emerge from the woodwork in droves. The market will settle. Some of the incumbents will survive, some of the new entrants will succeed, and most of the customers will (eventually) work out what they really need. Visualisation’s equivalent of using a different font for every paragraph (just because you can) and the <blink> tag will fade, just as it did in desktop publishing and web ‘design’.

The new data visualisation tools from companies like chart.io demonstrate an interesting trend toward atomisation of business tasks. We’ve seen this in the consumer space for a long time, particularly since the advent of channels like Apple’s App Store. But in business, the trend has been towards consolidation, with fewer and fewer software tools trying to do more and more. These bloated behemoths become ever-harder to use, ever-harder to support, and developers find themselves increasingly constrained by dependencies and legacy as they try to innovate or encourage the next upgrade cycle.

Now, though, we’re seeing renewed enthusiasm for the small, focused, application. It needn’t do much, but it must do what it does compellingly and well. Increased provision of APIs helps, of course, with users more easily able to extract data from a source, work with it, and then pass the result on to the next application in the chain. In principle, a recipe for freedom of choice, best of breed, and healthy innovation. Whether it continues to work in practice remains to be seen.

Many of these topics surfaced in a conversation earlier this week, as I spoke with Vik Singh, CEO and co-founder of Infer. The company has been in the news this week because it secured $10m in new funding. But Singh talks passionately of the need to “do one thing well.” Infer, he suggests, does that. They’ve identified a problem — lead scoring — in which the focused application of some effort can quickly lead to a “measurable lift on the top line” of their customers. Infer can be deployed (as SaaS) in a matter of days, integrated with existing tools such as Salesforce and Marketo, and quickly start to deliver demonstrable returns.

We focus on a particular problem… Our team understands what matters. We don’t focus on counting features. We focus on finding the least amount of features to deliver the most value.

It’s a refreshing attitude, but does it create a product — or a company — that can grow? Singh certainly seems convinced that it does, and talks of ways that the basic notion of lead scoring can be extended as customers move away from intuition and best-guesses towards a data-based decision making process.

Tools are increasingly emerging to help business decision makers be better informed by the data that matters to their organisation. Some of those tools are best used by people with the technical and business skills to understand data and its implications. Some are, perhaps, too easy to use, and will encourage half-baked analyses based upon inappropriate data. In purely Darwinian terms, those tools — and the companies that invested too heavily in them — will most likely fall by the wayside. We’re at an early stage in a journey that needs to see all of us become more data literate. Good tools, smart companies, and some great lighthouse examples can all help that. Just as use of desktop publishing and web design tools settled down, we’ll see the same with data manipulation tools. I’m not particularly worried about this industry’s ability to deliver good infrastructure tools, good data ingest, storage, and manipulation tools, or good data visualisation tools.

I’m more concerned to ensure that we nurture the softer skills. Almost anyone can make a chart (they teach it to 7 year olds!). Many people can even make the right chart. Far fewer people can craft the narrative that makes the data in that chart sing, that uses it as a tool to persuade, to shape, and to take decisions. We’re cracking the technical side of the problem. Now we need to find — and to celebrate — the story tellers.

And just to finish off, it was interesting to see Stephen Wolfram‘s blog post this week, Data Science of the Facebook World. Lots of data, lots of graphs, and the beginnings of a story to make the data sing…

Image, produced by Mapumental, shows average public transport journey times into central Edinburgh.

Related articles

• Forget big data, small data is the real revolution (guardian.co.uk)
• 10 Reasons Why We Visualise Data (ritholtz.com)
• Discussion: Storytelling and success stories (visualisingdata.com)

During Monday’s Analyst Day, and in sessions throughout the conference, we began to see evidence of fledgling steps being taken beyond the early adopters on the bleeding edge. As Hewlett Packard’s Florian Otel remarked in his session, “it’s called the bleeding edge for a reason.” Some people want to be there. Some people need to be there. The majority of adopters, however, simply want robust, credible, dependable, and wide-ranging solutions that work. After a period in which OpenStack’s hype, perhaps, outpaced its reality, might we finally be seeing some credible proof points beginning to emerge?

The fledgling OpenStack Foundation clearly recognises its own need to move beyond an introspective community of tech enthusiasts playing with an interesting new toy. The video (embedded below) which loudly greeted the audience for Tuesday’s opening keynote was not subtle. If OpenStack is to succeed, the e.n.t.e.r.p.r.i.s.e is where it must go.

Announcements and briefings during the event definitely lacked the wow factor, but that is not unexpected at this stage in the project’s evolution… and may even have been intentional. The message was one of steady progress (a 26% increase in patches to the code since Q4 of last year), broadening community (a 27% growth in participating companies), and growing interest (a doubling in website traffic).  Beyond some lighthouse deployments at Best Buy, Bloomberg, Comcast, the National Security Agency and the like, it was harder to assess the extent to which OpenStack code is actually being used. Josh McKenty of Piston suggested that there were around 400 production deployments of OpenStack, plus 20 times that number of pilots. Amazon, again and again, was the competitor to beat, which may be a mistake. As Cloudability‘s Mat Ellis reminded me, this really isn’t a zero sum game. OpenStack doesn’t have to take users away from Amazon to win. There is a massive opportunity in greenfield deployments, and in helping customers to migrate from more traditional on-premise IT facilities. OpenStack may be better served by looking ahead, rather than always benchmarking itself against the Seattle behemoth. Yes, they compete to a degree. But there’s still plenty of room for both.

More worryingly, there was clear hubris with respect to OpenStack’s more obvious competitors; VMware, Eucalyptus and Apache’s CloudStack. VMware, I was told far too often, “cannot deliver a viable solution.” Eucalyptus was “dead in the water,” and CloudStack “irrelevant.” More positive — or realistic! — assessments appeared disconcertingly thin on the ground.

Amongst the case studies, perhaps the strongest was the one we’re not allowed to talk about. I hope that the OpenStack Foundation realises what a powerful story it is, and that they’re doing all they can to get the company concerned to permit its story to be told.

So, if the sessions and attendees in Portland are any measure, OpenStack is doing fine. It’s at a tricky stage in its evolution, though, as it moves from rapid development and developer evangelism toward dependable delivery and enterprise adoption. The first evidence of that shift is there to be seen. By the next OpenStack Summit (in Hong Kong), I’d hope — and expect — to see quite a bit more.

Disclosure: the OpenStack Foundation paid my travel and expenses to attend this event in Portland, Oregon.

Related articles

• At NSA, The Cloud Is About Big Data And Moving Beyond IT (techcrunch.com)
• OpenStack Summit Highlights: HP Support, Third Party Solutions Gain Traction (servicesangle.com)
• AWS vs. VMware vs. OpenStack: And The Cloud Winner Is… (readwrite.com)
• Top 5 lessons learned at OpenStack Summit [GigaOM] (gigaom.com)
• Rackspace Looks To The Telcos In Global Expansion Of Its OpenStack Cloud (techcrunch.com)
• Shuttleworth: Canonical is fast and easy with OpenStack clouds (go.theregister.com)
• OpenStack gains momentum as vendors give Grizzly a bear hug (infoworld.com)

The cloud powers my business. The cloud is what I talk to clients about, it’s what I write about, it’s what people pay me to know about. The cloud (and, more generally, the web) make it possible for me to work with clients around the world, often without leaving a small market town in the East Riding of Yorkshire. Every piece of software I buy, install and use (except Microsoft Office, which I do still have to endure from time to time) is little more than a window onto the cloud. Most of those software tools have a web interface that I could use. Even tools which don’t (like OmniFocus) have competitors that do. So in principle I could do pretty well all of my mainstream tasks in a web browser. But I don’t. And I’m not yet sure that I want to.

On one level, mainstays of my working day like Evernote, Reeder and Dropbox exist to ensure that the content I want is available on whichever device I’m using, whenever I need it. Desktop, laptop, tablet and mobile phone all access the same shared pool of content, banishing the old world in which either everything travelled on a USB stick or (more likely) the document you wanted was on your other device. Both Evernote and Dropbox (and other tools I use) have decent web interfaces. I could just use those, and (provided there’s a network connection) realise the same basic benefit of access to documents and data. But the desktop clients (and mobile apps) bring a degree of polish to the user experience that – to me – still seems to add real value. If nothing else, local caching makes them quicker. Indeed, Reeder is really just a local window onto a web-based service (the soon-to-die Google Reader). That dependency creates an interesting challenge for Reeder’s creator, following Google’s decision to shut down their service.

For email, calendaring and basic document creation, however, I’ve fully embraced the browser-based experience. Google Apps, running inside Chrome, with a few core extensions such as Rapportive. Here, locally installed software no longer adds any value for me. Blogging, too, mostly happens inside a WordPress editor these days. Offline editors like MarsEdit, although good, now languish unused on my hard drive. The boring financial side of my business is also entirely on the web, with everything handled on my bank’s web site or within FreeAgent‘s web-based tool.

And then there’s Skype. With no sign of Google Voice in Europe, Skype remains the mainstay for voice and video communication. And it only works if you’ve downloaded and installed the software. It would be difficult to rely upon a device which wouldn’t let me use Skype.

It’s certainly possible to do almost everything without locally installing any software, and that’s a remarkable step forward. But it’s really not clear that the web-only experience is good enough (yet) for people to embrace it by choice. Thin clients (like the Chromebook) may be cheaper than their Windows and OS X-powered equivalents, especially if sales volumes grow. They may be easier to manage, which must appeal to enterprise IT managers. But would an individual choose to buy one, except to save themselves a bit of money? Not yet, I suspect.

I’ve moved entirely from local email, calendar and blog clients to the web. I’ve moved mostly from local word processing to the web. How long will it be until the web interface for other services becomes our first choice, rather than a useful backup in those situations where you’re borrowing someone else’s computer? It will be interesting to see… The Chromebook (both the eye-wateringly expensive Pixel and cheaper variants) offers an interesting illustration of future potential. With the current state of web tools, though, today’s Chromebooks cannot be more than a niche play. It will not be many years before that changes, and light, fast, cheap, well-connected devices with great batteries become a valid choice for the majority of users in need of a device that isn’t a tablet or a smartphone.

Image © Google.

Related articles

• Google’s Chromebook Pixel: $1,299 for a freaking touchscreen Chromebook (venturebeat.com)
• Google unveils touchscreen laptop (bbc.co.uk)
• First real-world usage figures suggest Chromebooks are struggling (zdnet.com)
• My month with the Chromebook Pixel: A review (cnn.com)
• Naysayers be damned: Why I bought a Chromebook Pixel (gigaom.com)