Image via Wikipedia

Last week, open standards body OASIS unveiled yet another shiny new standards effort. The OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) Technical Committee hopes to make it “easier to deploy cloud applications without vendor lock-in,” and to support moving from one cloud to another. The usual suspects — the likes of IBM, CA, and Cisco — are on board. The usual holdouts — Google and Amazon, of course — are not. So what is TOSCA trying to achieve? How does it fit alongside all the dead, dying, or ponderously deliberating cloud standardisation efforts that have gone before? And without the giants of the cloud, is there really any point bothering?

As I’ve probably mentioned before, involvement in various national and international standardisation efforts played a big part in my early career. I went to the working group meetings in odd (but often beautiful) locations. I participated in the conference calls. I engaged on the mailing lists. I drafted and edited and reviewed the documents. I completely buy into the idea that there is a place for foundational standards, developed through consensus-building and maintained for the long haul by organisations that stand apart from the vested interests and their competing agendas.

I also believe that there’s a time and a place for these standardisation efforts. Do it too soon, and we end up ossifying something that needs to be in a state of flux. When you don’t know what the best way to prepare a meal is, it’s too soon to print the recipe book. We need to try different approaches, and we need to be able to throw away the attempts that didn’t work out. More worryingly, standardisation efforts can be used for political ends. They can be little more than a rod with which to beat the (usually dominant) competition. At best a distraction, or a talking shop for those unwilling or unable to just get on and do something. At worst, one amongst a toolchest of dirty tricks in a broader war for hearts, minds, and — ultimately — wallets.

The cloud market is a fascinating place. There are leaders and there are followers. There is innovation, and there is competition. There is agreement, and there is debate. For all the rhetoric, and all the posturing, we really don’t yet know the right answer to many of the cloud’s questions.

Maybe TOSCA and the Open Data Center Alliance and IEEE and the rest are — still — too early, and should be content to let the market thrash out a few more of these issues before anyone tries to write anything down? And when it is time to write some stuff down, let’s make sure we focus on specific, finite, tangible, atomic tasks rather than “the cloud.” As Dave Roberts commented in regard to TOSCA’s scope;

“That goal is so large, that I think it’s probably unbounded. When problems get unbounded, the best you can ever hope to achieve is to solve a large enough subset of the problem that the solution is still interesting. If you can’t achieve that, people ignore the solution because it fundamentally doesn’t help them. There is always an ‘interesting’ part of the problem space that they have to solve a different way, and that undercuts the use of the partial ‘solution.’”

And as for Tosca? Things didn’t end well for her, did they? Might TOSCA’s fate, too, be sealed?

Related articles

• Tech giants back standard for cloud portability – InfoWorld (infoworld.com)
• Proposed spec aims to nix cloud lock-in (gigaom.com)
• On TOSCA and Cloud Standards. MyPOV (diversity.net.nz)

Image of Stephen Wolfram via Wikipedia

British-born computer scientist Stephen Wolfram sees ongoing efforts to extend the Internet’s top-level domains (TLDs) beyond the familiar .com, .org, .uk etc as an opportunity to raise the profile of machine-readable data. In a blog post published yesterday, he argues that a new .data domain would increase “exposure of data on the internet—and [provide] added impetus for organizations to expose data in a way that can efficiently be found and accessed.” Whilst wholly in favour of Wolfram’s stated aim, I can’t help feeling that his suggested solution is at best unnecessary and at worst a worrying segregration of data from the ‘proper’ web that everyone else will continue to exploit.

Back in June of last year, the body responsible for coordinating the global domain name system approved a plan to permit new top-level domains (the letters after the final dot in an internet address — the .com in cloudofdata.com, the .uk in bbc.co.uk, the .edu in harvard.edu). Until recently, these top-level domains have been tightly controlled, with a small set of generic domains (.edu, .gov, .mil, .org, etc), a larger set of country domains (.uk, .fi, .nz, etc) and one or two others such as .eu. From tomorrow, anyone with $185,000 will be able to submit a proposal to create and manage a new top level domain, and it’s possible that there could eventually be thousands of them. Wolfram is keen to ensure that data doesn’t miss out on the ‘opportunity.’

As Wolfram himself recognises, there is already an awful lot of machine-readable data on the web. Some of it sits embedded within the web pages that humans read, with specially formatted code waiting to be triggered by the calendars, the address books, or the browser plugins of site visitors. Some of it is packaged up in data files, offered for download. And some of it waits inside a database, ready to be delivered in response to an API call or a query typed into a web form.

There is a growing enthusiasm for exposing this data for reuse. Government transparency agendas have driven public sector data sites like data.gov.uk and data.gov. Similarly, efforts such as data.open.ac.uk and data.southampton.ac.uk see universities beginning to consciously collect data sets together and offer them up for reuse. Similar efforts in the commercial world are less easy to point to, but that reticence has nothing whatsoever to do with the lack of a ford.data, boeing.data, ge.data or astrazeneca.data domain!

In some ways, the convention for gathering significant chunks of data on a data.xxx.yyy site echoes Wolfram’s intention, but with a number of advantages. Data without context is far less valuable than data with context. Much of that context may be inferred from the domain in which the data lives, with data delivered from a .gov or .edu (or .gov.uk or .ac.uk) site perhaps interpreted differently to data hosted on .com, .biz, or .xxx. Southampton University, the Open University, and the US Federal Government are able to gather data up and make it available for download via their existing data. sites if they choose. This offers human visitors to their sites a degree of convenience, whilst retaining the power and brand attributes of their existing domain. Gov.data, gov.uk.data, open.ac.uk.data, southampton.ac.uk.data, though? All are messy, in ways that Wolfram’s own wolfram.data would admittedly not be, and all are simply additional registrations that the institutions would have to pay for in order to stop someone else grabbing the domain.

At the end of the day, the machines don’t actually care. The existing data.open.ac.uk-type sites are human conveniences, not machine enablers. The computers, and the software they run, are quite capable of crawling the public web and finding accessible data wherever it lies on a site. There are plenty of reasons to continue embedding little snippets of data inside human readable web pages, regardless of whether you have a data.wolfram.com or a wolfram.data site. Content negotiation is becoming increasingly capable, such that there really is no need for what Wolfram calls a ‘parallel construct to the ordinary web’ at all. A human being arriving at a web site sees human readable content, whilst various software tools would automatically be presented with very different data or functions, optimised to their capabilities and requirements.

By all means, let us show the curious some of the existing techniques that work in making data more easily accessible. By all means, let us identify the gaps, the issues, the problems (none of which a new TLD even begins to address). Yes, let us definitely and unambiguously set about “highlighting the exposure of data on the internet—and providing added impetus for organizations to expose data in a way that can efficiently be found and accessed.”

But please, let us not be distracted by the false hope that adding yet another TLD to the babel that ICANN is about to unleash can do anything more than consign data to some online ghetto, wallowing unwanted, unloved and unused as companies and their customers lavish love, attention, and clicks upon the .com domain over on the ‘proper’ web.

Thanks to Raphaël Troncy, whose tweet first drew the story to my attention.

Related articles

• Is It Time For Computers To Have Their Own .Data Domains? (techcrunch.com)
• ICANN Pushes Ahead With January 12 Launch For New Top-Level Domains (wired.com)
• The biggest change in DNS since Dot-Com (wired.com)

Image via CrunchBase

I am in Madrid this week, as a guest of Hewlett Packard. The company’s European roadshow, Technology@Work, is in town for two days, showing existing and prospective customers some of the ways in which HP’s Converged Infrastructure can help them meet the growing challenge of continuing to grow IT capacity as data centres fill, power grids reach capacity, and budgets fall.

A few of us took the opportunity to sit down with Iain Stephen (VP Enterprise Storage and Servers, UK & Ireland) and Martin Riley (Programme Manager for Technology@Work) from HP yesterday evening, and the result is now available as a podcast. Also participating in the conversation were HP’s Becca Taylor,  Martin Macleod, Massimiliano Galeazzi, Patrick Husband, and Hans Vredevoort.

The conversation was recorded on a shiny new Ivy Worldwide Blue Snowball microphone that none of us had ever used before, in a room that was a little bigger than we’d have liked. All in all, though, I’m pleasantly surprised by the result. Have a listen, and see what you think.

Disclosure: our expenses were covered by Ivy Worldwide, on behalf of HP.

Image via CrunchBase

With apologies to Zanussi for the corny title, I had an interesting conversation with Axcient CEO Justin Moore and HP’s VP for Channel Strategy & SMB Meaghan Kelly about the issues of helping small and medium businesses cope with backup, disaster recovery, and business continuity.

Yesterday’s conversation was taking place in the context of today’s announcement from HP and Axcient; Axcient’s US data centres are moving to HP servers, networking and storage, and the appliance that the company puts on-site with customers will be an HP ProLiant server moving forward.

According to Moore, most SMBs use “4 or more” vendor solutions to “cobble together” a data protection solution. Basic application and data backup, disaster recovery from off-site backup, and business continuity in the event of hardware failure are all addressed separately, and it can be difficult to stitch the pieces together without dedicated — and expert — effort. I wonder how many companies, especially at the ‘S’ end of ‘SMB,’ simply give up on attempting anything but the most rudimentary on-site backup… and hope that nothing goes wrong?

Axcient’s solution is different, as the company offers a single appliance (an HP server, running Axcient’s applications) that can be deployed locally. There is no up-front cost for the appliance, with both it and the ongoing service being billed on a recurring monthly subscription. Software on the appliance handles backing up applications and data on servers, desktops and laptops across the business, and securely replicates changed data to one of Axcient’s US data centres for backup and disaster recovery. Should critical hardware fail within the business, a virtual machine can be started on the appliance to fulfil the role of the failed equipment until it is repaired or replaced. Indeed, Moore suggested that the ProLiant server they’ve selected would be capable of taking the place of up to seven servers if required; presumably with a perceptible loss of performance. Slow responses from a straining appliance are no doubt preferable to no response at all from seven dead servers.

Axcient’s data centres are currently in the United States, raising the spectre of PATRIOT Act, European Data Protection legislation, data territoriality and more for those of us outside the US. Moore recognises this issue, and says that the company is in the process of opening a Canadian data centre. Sites in Europe and elsewhere may well follow.

Both HP and Axcient are pitching this solution to the entire SMB market; that’s everything from the lone consultant operating out of a home office up to businesses employing 1,000 staff. They’re certainly right to address the whole market and the top end is probably more lucrative, especially for HP’s channel partners. I was particularly interested, though, in the value that this might deliver to small businesses struggling to do much more than replicate core data to an external hard drive under their desk. Here, the $150 per month ballpark that Moore mentioned is expensive, but perhaps a sound investment if data and systems are key to business success.

With their reliance upon asynchronous data connections and effectively consumer-grade connectivity, one significant issue for small businesses is the practicality of trickling all of their on-premise data slowly up to an Axcient data centre before they can begin to reap benefits. With their ‘rapid seeding service,’ Axcient appear to have thought of this. If connectivity issues mean that network-based methods to replicate all your on-premise data are going to be too slow, it’s possible to transfer the data to a storage device attached to the appliance, “ship that in a locked container” to an Axcient data centre, and have the data loaded there to be available within 48 hours. The incremental backups of new and changed blocks then take place over the network without causing issues. Simple, obvious, but a sign that Axcient appears to be considering the needs of their smaller customers.

Cukier – Strata so far been focussed on the tools and toolmakers. Real-world applications will actually lead to biggest changes, but we haven’t really heard from them here. Big Data – big, fast, smart, messy.

Varia – Big Data Clouds. 3 case studies from AWS customers. Razorfish… cost of storing and analysing data. Best Buy wanted to analyse clickstream logs and predict patterns etc to aid advertising placement. 3.5bn records, 71 million cookies, 1.7 million targeted ads per day. Using AWS allowed move from upfront $500k CapEx to $0, ‘significant’ recurring OpEx to manageable costs, etc. Led to 500% increase in return on ad spend… but led to huge data quantities.

Yelp – 8 countries, 50 cities, 100GB of logs per day… 200 Elastic MapReduce jobs per day on Amazon, processing 3TB of data.

Etsy – >500GB web logs per day, 400K sellers.

Use cases useful… but I’d rather have heard them from the customer than from Amazon.

Storage getting cheaper. Analytics getting faster. Analytics getting smarter…

Pereira – ‘what happens when you turn on the lights?’ Capacity planning etc an increasingly complex issue for power companies. Smart algorithms, cloud computing and big data analysis creating opportunities to get better at these decisions. Deep modelling of resources. Model ‘every single generator in the country,’ etc. These models ‘give better results, because the situation changes dynamically.’

Hurwitz… big data ‘has been a game changer in the legal industry.’ ‘It’s a necessity if your legal department is to be successful.’ When a company gets sued, it goes through a discovery process; it gathers up all the relevant information and passes it to lawyers on both sides. It’s a complex process… and there are consequences if you miss things. SOLR, Lucene, Hadoop and a range of other tools used by legal teams to build searchable pools of the relevant data. Back-end works, but UIs struggle. Machine learning has been applied to classifying documents more quickly, accurately and affordably.

Cukier – data as by-product? Everyone using data internally, and using Big Data to do new things with old data. Has anyone taken data they’ve generated, used it internally, and then realised that it has tangible external value?

Pereira – Colombia sells their database to investors overseas. Energy companies exchange raw data in a marketplace.

Varia – amazon.com and amazon web services see value in DaaS. eg amazon.com products have lots of associated metadata… which is available through an API. Also companies like 80legs crawling the web, storing the data on AWS, and selling the crawl results to third parties.

Questions – how do you get a community (like legal) comfortable with using these tools?

Hurwitz – legal community was resistant. Economic reality that they had to embrace these tools to enable court-mandated e-discovery. A large case could include costs of millions of dollars, just to gather data before the case begins.

The notes that follow are a rather raw summary of some of the things I’m hearing. Later in the week, I’ll try to come back and extract the main issues in a rather more polished form.

Up first, Mike Driscoll of Metamarkets, talking about Mining the Tar Sands of Big Data.

Trying to frame conversation; why are we all here? Why is now the time that we’re all beginning to focus on ‘Big Data’ ? Drawing analogy with the tar sands in Alberta; lots of oil there, but it’s been expensive to extract.

If ‘information is the oil of the 21st century [as Gartner has suggested], then Big Data are the tar sands;’ lots of data, sitting in our data centres, waiting for us to invest in extracting usable data. Expensive, painful, but ultimately valuable.

‘Attack of the exponentials;’ cost of storage, bandwidth and compute falling exponentially. Number of nodes on the network rising exponentially. Intersection creates ‘data singularity.’

Unlike oil, data abundant and renewable. Like oil, extraction of data creates value. Cheaper and easier to extract value from data than ever before.

Three forces reshaping data landscape; sensor networks, cloud computing, and machine learning.

Sensor networks; now prevalent, all-pervasive, ubiquitous, and typically connected. Generating vast amounts of data.

Cloud computing; ‘used to mean everything and nothing.’ But real advantage is that it turns CapEx into OpEx.

Machine learning; gives us the capabilities to process the flood of data, intelligently. Smart Planet, Smart Grid, Smart Business, etc. Driverless cars, spam filters, recommendation engines all drawing upon Machine Learning ideas. All iterating and improving with increasing rapidity. Unlike Cloud and sensors (which are becoming commodities), machine learning algorithms are – and may remain – a competitive advantage.

Four consequences of all this; battle for finite number of good data scientists, changes in the way that data is published (and valued), the end of privacy (?), the rise of data startups.

Battle for data scientists; difficult to hire people who can munge, interpret, and tell stories with data. And everyone last night at bigdatacamp was ‘hiring’.

Retailers, banks, online publishers, etc have tended to hand over the keys of data management to third parties. Seeing pendulum shift the other way, as companies recognise the value of their data and seek to control it – and realise the value. Tension with ‘open data,’ data to the cloud, etc?

Privacy – not about shifting access to data, but about more accurately defining the ways in which it may be used.

Q&A

‘What sort of educational background does a data scientist need?’

knowing some stats helps. knowing some programming helps. But curiosity is key. Not sure there’s a degree out there. Pick up the skills if you have the right mindset. ‘You have to be a bit of a hacker,’ says Mike.

‘What are the three big problems that data science will solve?’

Making sense of the world around you; Freakonomics, for example. Taking data and making sense of how the world is working. Scaling up decision making, so that a data-powered story can be presented in a way that lets people make intelligent decisions.

Next, Barry Devlin from 9sight Consulting talks about The Data-Driven Business and Other Lessons from History.

‘the old guy who has been brought along to talk about history,’ and ’illegitimate grandfather of data warehousing’

Address Past, Present, and Future.

Past – the origins of Data Warehousing

Data Warehouse architecture work at IBM in Europe in the mid-80s.

‘Big Data’ (a couple of hundred MB, at the time) created need to structure the Enterprise Data Warehouse in a particular way. Led to silos. ‘Hard information’ only, at the time. Warehouse designed in a well-architected fashion. Ensures that data flows in a single direction. Possibly too regimented for the 21st century?

Lessons?

Information quality and reliability are key; Master Data Management, etc. This is unlikely to change.

Data volumes and variety have presented big challenges over the years. Expectations and business demands outstrip technological capabilities. Organisational and political issues hamper progress. This is unlikely to change.

Exploration and analysis drives innovation.

Present – Business and technical challenges

3 key trends in business are driving rapid change; closed loop business, massive information volumes, collaboration driving innovation.

Really important to stop talking about ‘unstructured information;’ that’s just noise. Information has structure. Instead, hard information is data; tables, structure, computer-oriented. Meaning and values have been separated. Metadata explicit, and formally modelled. Soft information is not well defined, it is by and for people, it mixes meanings and values. Metadata is implicit, tacit, or non-existent.

Moving from information we understand – and control – to information outside the enterprise that we don’t. Implications for quality, meaning, etc.

Future – a new architecture?

current architecture 25 years old – time for a change?

Next, Bob Page from eBay, talking about Building the Data-Driven Organisation.

eBay “fascinated with numbers;” early 1999 screenshot of homepage, showing lots of stats.

What role does data play in the business? Using Analytics focussed on big buckets; velocity, efficiency, trust, etc.

Efficiency drive – lower insertion fees to list/sell new products by 99%. Decision based on analysis of data?

Trust – top-rated sellers constantly monitored to ensure algorithm is reflecting reality. 22% of sales from trusted sellers in 2009. Now 32%. Actually surprised it’s not higher…

eBay handled $2Bn of sales on mobile devices last year.

Analytics and continual data analysis drives all the apps, trust metrics, etc.

Some figures… 50 TB/day of new data, etc. Lots of other numbers on slide, but it was only up for seconds…

Analytics work in marketing, sales, product dev, and all areas of the business.

More than 85% of eBay’s analytial workload is new and unknown; design for the unknown. Enable exploration of the data, rather than just reporting of established metrics.

Machine Learning: Data trumps Algorithms. This is the promise of Big Data; existing algorithms get better as you throw more data at them. It’s cheaper to throw more data at an algorithm than to invest in developing new algorithms.

Enterprise Data Warehouse for transactional data; purchase history, etc. Behavioural data to track clickstreams, impulse purchases, etc… much larger than the amount of transactional data in traditional systems. No technology silver bullet for the behavioural data; optimise for concurrency, or TCO, or CPU usage, or flexibility, or storage, or governance? Those priorities change the tool you should use.

eBay built a 500-node Hadoop cluster in June 2010. Now they have a much bigger cluster.

Data Marts; ‘a reality for many of us,’ because businesses need to give control to the user. Don’t want infrastructure team/ data scientists as bottle neck. Totally opposite to attitude expressed by previous speaker. But Data Marts end up being very expensive and inefficient. eBay have built a virtual data mart; views onto a single pool of data. Far more efficient, in theory.

And then I had to slip away, and miss the final session before lunch…  More later…

San Jose-based Cirtas emerged from stealth back in September 2010 with a $10 Million (€7.86 Million then) Series A funding round, their novel Bluejet hardware appliance, and the backing of Amazon.

Today they’re back, with a new CEO and another $22.5 Million (€16.6 Million) in the bank. The Series A investors — New Enterprise Associates, Lightspeed Venture Partners and, unusually, Amazon — are joined by Shasta Ventures and Bessemer Venture Partners for a Series B round that positions the company for some rapid growth.

Cirtas’ Bluejet Cloud Storage Controller is a hardware appliance, deployed in the data centres of medium and large enterprises to simplify the task of integrating existing on-premise Tier 2 and Tier 3 storage with disparate Cloud-based solutions such as Amazon’s S3, EMC Atmos, Iron Mountain and AT&T’s Synaptic Storage as a Service.

Talking ahead of today’s announcements, new CEO Gary Messiana suggested that Cirtas’ decision to deliver a hardware appliance rather than a software-based solution reflects their deep understanding of both their customers and the sales channel. Messiana is not the first to suggest that buyers of enterprise storage are a conservative bunch, and he’s certainly not criticising that conservatism; when business continuity depends upon the decisions you make and the systems you buy, you’re hardly going to take unnecessary risks, now are you? A piece of physical hardware that you can see, touch (and even hug) delivers an element of familiarity that appears to appeal to enterprise-class customers taking the first steps to leverage Cloud-based storage within their existing solutions portfolio. With Bluejet, Messiana suggests, control continues to reside inside the data centre. The Cloud provider(s) to which the appliance directs data are simply (dumb?) utilities upon which the enterprise may choose to draw in a manner abstracted by Cirtas’ technology. A hardware solution also suits the channel-based (rather than direct sales) model by which these companies tend to buy. As Messiana notes,

“[much of] the money changes hands in the channel… and [channel partners] know how to sell tin.”

Formerly an entrepreneur in residence at Series B participant Bessemer, Messiana talks persuasively about the clarity of proposition and go to market strategy that drew him to Cirtas. Drawing upon pre-Bessemer experiences as CEO of traffic optimising Netli (acquired by Akamai in 2007), Messiana argues that the shift off-premise makes expertise in moving data just as critical as the data storage skills of incumbents. Might Akamai, Limelight and their ilk make inroads into this market, at the expense of EMC, HDS, 3Par et al, I wonder?

In storage as in so much else, big incumbent enterprises are very different from smaller or younger companies. Whilst startups and SMEs might be quick to embrace entirely virtual solutions — often starting in the Cloud rather than migrating to it from elsewhere — the ‘multi-billion dollar corporations’ served by Cirtas will almost inevitably follow a very different path. Across Pharma, HR, manufacturing, publishing, insurance and finance, Messiana reports that customers with market caps of $500 Million – $10 Billion and more are flocking to the company.

So why take more VC money, so soon, and dilute the company? Messiana insists that “plenty” of the initial $10 Million is still in the bank, and that VCs were falling over one another in their enthusiasm to invest. The deal was apparently closed rapidly, with an aggressive valuation that sees “minimal dilution” whilst giving Messiana the cash to expand sales, support, and other areas of the company.

Cirtas would appear to be off to a good start, but it would be dangerous to be complacent. The company is not alone in seeing hardware as a way to encourage enterprises toward the Cloud, and there are plenty of software-based cloud storage gateways waiting for the opportunity to demonstrate their capabilities as those conservative CIOs become more willing to trust the Cloud. And then there’s Amazon. What might they do next?

Related articles

• Nasuni grabs cloud development cash (go.theregister.com)
• Enterprise Poll: Do You Use Cloud Storage? (readwriteweb.com)
• F5 Friday: F5 ARX Cloud Extender Opens Cloud Storage (devcentral.f5.com)
• The $20 Million Club: 10 Well-Funded Cloud Startups (nytimes.com)

I’ve been a fan of Om Malik‘s boutique analyst site, GigaOM Pro, pretty much from the outset, and happily renew my subscription each year. The site covers a wide range of industry topics, and those Quarterly Wrap-ups are worth the fee all by themselves.

I’ve written a few reports for them in the past, but was delighted when Mike Wolf got in touch to see if I fancied trying my hand at curation on their Infrastructure/Cloud channel.

So next week (from 31 January) I’m going to be gathering and commenting upon links from around the web, writing a daily ‘Today in Infrastructure,’ and finishing off with a Weekly Update. If you’re not (yet!) a subscriber, why not sign up for a free seven day trial and join me for the start of my little adventure?

If there’s something you think I should be covering, do let me know.

Related articles

• Free GigaOM Pro Webinar: The Scalable Cloud (gigaom.com)
• GigaOm raises $2.5M, claims 10,000 Pro subscribers (venturebeat.com)
• The Cloud has a place, even inside heavily regulated industries (cloudofdata.com)
• Talking Scalable Clouds with GigaOM Pro and Limelight Networks (cloudofdata.com)
• GigaOM Structure 2011 (datacenterknowledge.com)
• Join GigaOM at Big Data on March 23 in New York City (gigaom.com)

Image via Wikipedia

As part of a restructuring of the organisation’s US-based data centres, the acronym-laden NATO (North Atlantic Treaty Organisation) today announced that its ACT (Allied Command Transformation) will be deploying IBM‘s PCS (Private Cloud Solution*) inside its HQ SACT (Headquarters, Supreme Allied Commander Transformation) in Norfolk, VA.

According to IBM’s E.J. Herold, the company is delivering the software component of their ‘Cloud in a Box’ solution. Normally sold alongside racks full of IBM blades, the NATO installation will see IBM’s software running on the data centre’s existing (mixed) hardware. A second extant data centre – in San Diego, CA – will also be shut down as part of the transformation.

Comprising 28 members from Albania to the United States, NATO constantly grapples with issues of interoperability. 28 members, plus NATO itself, means (at least!) 29 different IT infrastructures, (at least) 29 different communications protocols, and (at least) 29 different views on the standards and specifications to underpin any one IT deployment. In operational environments, whether military or peace keeping, limitations in interoperability of IT inevitably lead to expense, confusion, delay… and perhaps loss of life. Following NATO’s summit in Lisbon last month, the organisation reaffirmed the importance of greater cooperation between partners.

Today’s announcement aligns well with this new ‘Strategic Concept,’ delivering a cost-effective test and development environment to the ACT. According to IBM’s press release,

“The on-premise cloud will be used to test and develop network solutions for command, control, intelligence, surveillance, and reconnaissance projects. The goal is to demonstrate how recent developments in cloud computing can reduce ramp-up time for enhanced technology capabilities, while improving important operational functions, such as increasing situational awareness and faster decision-making.”

In this initial test/dev deployment, it is anticipated that IBM’s solution will enable NATO to more rapidly explore a range of scenarios for collaboration and data exchange across the Alliance. Broader roll-out of the Cloud solution itself may follow in due course, initially to other NATO sites, but ultimately into the military infrastructure of NATO’s members. IBM’s Herold suggests that, with this project, NATO ACT is taking a leadership role in developing techniques and solutions that will make NATO itself more effective… and eventually trickle down into the internal practices of NATO member states.

Here, as in so many other markets, the relative safety of test&dev is an obvious place to begin exploring the potential – and pitfalls – of a Cloud solution. As more data centres – and more partners – become involved in future deployments, the ugly spectre of security will of course raise its head. But if NATO and its members can (mostly) keep their existing infrastructure secure, a private cloud should be no more risky…

Today’s announcement is relatively modest, and many purists would argue that it’s not even a Cloud. But it’s a tentative first step for a security-conscious organisation made up of (at least) 28 frequently paranoid military hierarchies, and will become far more interesting – and challenging – as it expands beyond the (relatively) safe confines of a single data centre inside a US Naval Base. Definitely one to keep an eye on.

* ok, I made that one up.

Related articles

• NATO, IBM team up on cloud, collaboration project (zdnet.com)

I don’t think I’ve ever really been convinced by the argument that everything will end up in the data centres of Amazon.

The straightforward provision of commodity Cloud Computing is an important – and growing – area, and one that will continue to expand as interfaces become simpler, FUD is challenged, and prices maintain their relentless march towards the bottom. Everyone has something they could usefully, sensibly, and cost-effectively run in a commodity Cloud such as those offered by Amazon, Rackspace, Flexiant, and others. In this space, basic stability, security and reliability combine with a compelling – and diminishing – pricing proposition to create commodity services targeted squarely to lowest common denominator functionality. Here, market forces may (inevitably?) lead to an eventual reduction in the number of providers. Cost, although not the only consideration, is both important and compelling. Although markets like competition, there may even be a single winner here, one day.

Layered all around the basic, routine, grunt-work computation that these commodity public clouds handle so well, many organisations find themselves having to cope with a wide range of other use cases and data sets. Some require specialist hardware (like the GPUs that Amazon has recently begun selling access to). Some demand particular regulatory and legislative hoops to be jumped through. Some have quirky requirements around latency in data transfer or speed of in-CPU processing. Some have lots of data, and issues with regard to getting the stuff from one location to another with a sensible balance between transfer cost and time.

All of these are certainly capable of being addressed in the Cloud, but the economics and the business rationale begin to shift. For the data owner, cost may no longer be quite so significant a factor. Reliability may matter more, or speed, or the audit trail. For the Cloud provider, these requirements no longer look like the lowest common denominator. It’s not cost-effective to provide these capabilities to everyone and still keep the price low. It becomes more sensible to segment, to divide, and to create bespoke offerings of various kinds. Some of these services require such specific things in terms of network topology, physical building layout, and staff expertise that it may even become counter-productive to have these services in the same building as the commodity Cloud. Here, there’s plenty of room for new entrants, plenty of scope for competition, and plenty of opportunity to differentiate in terms of price, location, support, and a host of other factors. This segment of the Cloud is only just getting started.

In these contexts, we see compelling arguments made for on-premise private clouds, off-premise private clouds, hybrid clouds, community clouds and the rest. Some of the arguments made in favour of private and hybrid certainly are part of the FUD we see in this space, but beneath the noise, the security scares, and the vested interests of SysAdmins and sellers of data centre components, there lies a grain of truth. Not everything is most sensibly run on a cheap VM, rented from Amazon (or Rackspace, or whoever) with your credit card, and physically located half way round the planet.

Unfortunately, it can be difficult to make sensible decisions about which type of cloud works best in each situation, and large swathes of the market are doing everything in their power to add to the confusion.

Having accepted that the basic offering from a public cloud provider is not the solution for my particular requirements, where do I turn next?

Do I listen to the (convincing) pitch from a vendor of ‘community cloud’ solutions for my domain? If I’m in Healthcare, they come with HIPAA and European Data Protection Directive, and all sorts of other accreditations. For dealing with sensitive patient data, this may be just what I need… but does the wily salesman also persuade me to run staff email and the hospital volleyball club website on this over-specified (and expensive) infrastructure?

Do I listen to the (convincing) pitch from a vendor of virtualisation software? If I’ve got a reasonably sized data centre with some life left in it, I may see the value of virtualising all of that expensive hardware, and running current applications in house more efficiently. But instead of gradually reducing my in-house costs, do I continue to add more machines as current ones reach end of life, or as new requirements come along?

Do I listen to the (convincing) pitch from my co-location facility, which happily sells me a ‘private cloud’ that may fail to deliver some of the economies of scale so central to the main Cloud proposition?

Do I listen to the horror stories, stick my head in the sand, and simply keep ordering servers until every single one of my competitors undercuts my costs and I go out of business?

These, and more, are certainly possible. But let’s return to that UK project I mentioned right at the start.

Flexible Services for the Support of Research (FleSSR) is

“a new cloud pilot project looking at utilising hybrid private-public IaaS cloud infrastructure to provide computational and data services to the academic research community. The project is a collaboration between the Oxford e-Research Center, IT Service @ University or Reading, e-Science Centre @ STFC, Eduserv, EoverI, Eucalyptus INC and Canonical Ltd.”

The ten month project is funded by the Joint Information Systems Committee (JISC), an organisation that supports the innovative use of IT across UK universities.

Now, to a degree, the project’s mindset must be influenced by its partners. IT staff at Reading and STFC are incumbents with turf to protect (or new vistas to discover, map, and claim). Eduserv has a new data centre that they’d like to fill with willing clients. It would be easy to be cynical, but knowing some of the people involved, I see no real reason to be. It is perfectly reasonable to suggest that a ‘community’ the size of UK Higher Education would realise value in replicating less (not nothing) at every university campus across the country, and bringing much of that together in some sort of Cloud. That Cloud might use public infrastructure, or it might be served up from an organisation such as Eduserv, which is known to the community, aware of the community’s requirements, quirks and foibles, and (importantly) not-for profit (and therefore cheaper?).

Personally, I’d always rather presumed that an organisation like Eduserv (or JISC itself) would act on behalf of the community to procure a competitive price on access to the resources of Amazon, Rackspace, or one of the others. I’m not convinced that most UK research computation needs any sort of special treatment that couldn’t be met from Amazon’s Dublin data centre… unless the community itself can somehow beat – and continue to beat – Amazon on price. Somewhat surprisingly, that’s exactly what some calculations in two posts by Eduserv’s Andy Powell suggest could happen. By including network costs and other charges over and above the basic storage cost, Andy finds Amazon, Rackspace and Dropbox to be more expensive than anticipated, and posits that Eduserv (connected to every UK university free of charge via JISC’s high speed JANET service, and constrained in the ways it can generate profit from services sold to universities by its charitable status) might actually work out cheaper.

There’s a lot of work to do in terms of fleshing out the assumptions behind some of Andy’s figures, but the whole industry certainly benefits when people conduct exercises like these out in the open, for all to see. If Andy has made mistakes, the vendors should be quick to jump in and correct them. If his assumptions miss the mark, public debate can redress the balance.

The Cloud is not all about price. But more transparency around the true cost of computing in the Cloud – and in your data centre – means that we can all make more informed decisions.

Thanks for sharing, Andy – and hopefully readers will be willing and able to look over your calculations and share their own views.

Note: this post was conceived and written in the United Kingdom. By reading this post you agree to comply with UK usage, and will henceforth pronounce the word ‘niche’ from the title as ‘neesh,’ not ‘nitch.’ Or maybe not.

Related articles

• Rackspace launches managed cloud services (zdnet.com)
• Are hybrid clouds the path to cloud-computing nirvana? (venturebeat.com)
• We’ll Take Care of Your Cloud, While You Manage Your Business (rackspacecloud.com)
• Trust Is Key For Cloud Success And What Can We Do About It? (cloudave.com)