Image: NASA

Amazon’s globe-encircling cloud infrastructure is compelling to many. From Virginia to California, from Ireland to Singapore, and from Japan to Brazil; wherever you find yourself there’s a local instance of the same familiar set of services. And, in all likelihood, Australia will soon be added to the list. For those primarily interested in just serving both Europe and the US, the list of options grows to include Rackspace, GoGrid, CloudSigma and a few others. And yet, despite the buying power and increasing ubiquity of these larger players, there seems to be plenty of space left for smaller entrants. For prospective customers only concerned with a single country or region, for example, the choices are almost too many to count, and choosing between them becomes a complex and multi-faceted affair.

Take Brightbox, for example. As my family know all too well, those pesky timezones mean that many of my evenings are punctuated with calls to or from the States, where so much of the innovation in this sector continues to take root and grow. Either that, or I’m creeping out of a sleeping house to catch early trains for the 150 mile journey south to London. It was therefore refreshing to talk to someone in this industry whose offices are only 50 miles away in the UK city of Leeds.

Established back in 2005 as a Ruby shop capable of hosting apps on dedicated hardware, Brightbox has evolved to place increasing emphasis upon the provision of infrastructure. In 2010, the company began to seriously explore the possibility of offering a generic cloud infrastructure environment. This was in the days before OpenStack, but Eucalyptus existed and was attracting interest. But according to Brightbox co-founder Jeremy Jarvis, 2010′s Eucalyptus lacked some key resilience features (load balancing, multi-data centre capabilities, etc) that the team believed were critical… so they built their own system from the ground up. And, at the end of September last year, Brightbox Cloud entered general availability.

The Brightbox cloud operates out of two UK data centres, with planning underway for a third. Both data centres are now owned and operated by Telecity, which acquired the two independent data centre providers with whom Brightbox had launched. Brightbox owns the racks and (Dell) hardware, and also ensures provision of redundant network access into the data centres. Customers are predominantly drawn from across Europe, but Jarvis says he’s seeing some customers coming from as far away as Australia, New Zealand, and Brazil. The company sees its potential growth through eventual expansion to data centres on the European mainland, but Jarvis says he’s “much less interested in setting up yet another” North American operation. The company is profitable, employs ten staff, and is seeing steady growth in usage.

Brightbox does not (yet) offer a web management console, but Jarvis describes this as a conscious decision and also something of an asset. The company has instead focused their attention upon crafting a rich, capable and intuitive API (and associated command-line interface). According to Jarvis, the developers that the company tends to target have responded favourably to the API, describing it as “nice” and “more consistent” than the various APIs offered by Amazon’s growing suite of services.

A focus upon developers (and the growing Dev/Ops movement) was also a strategic decision, and Jarvis cites examples in which ‘mere developers’ have proved instrumental in securing significant contracts with Brightbox from their employers. Corporate purchasing processes may, finally, be evolving. Despite the success of Platform as a Service (PaaS) offerings such as Heroku and EngineYard, Jarvis also suggests that Brightbox is seeing growing evidence that developers are seeking more fine-grained control over infrastructure than PaaS typically offers. A platform abstracts the underlying complexity of infrastructure, making it easier for application builders to focus upon creating the specific services they wish to provide. But abstractions typically require compromises, with configuration decisions being made for everyone on the platform on the basis of ‘normal’ requirements. For developers with non-normal requirements (and they may actually be the majority of users), IaaS is more likely to offer the fine-grained control that they need.

But the cloud infrastructure world has come a long way since Brightbox began planning their product two years ago. OpenStack has arrived, and (despite a growing body of nay-sayers) is credible. Rackspace, HP, and others are on the cusp of delivering real clouds to real customers on the back of its codeline. Eucalyptus appears to have turned a corner, and pulled back from a brink that I (and others) saw them teetering on the edge of. Canonical’s marriage of Ubuntu to OpenStack is now just one of several ways to get the same cloud code, capabilities and apis onto machines running inside your own data centre. Amazon just keeps on doing what Amazon does, incrementally adding features, cutting prices, and becoming ever-harder to not choose.

In that world, surely a little cloud provider operating their own bespoke solution from the wrong end of the Leeds-London railway line, in a country on the wrong side of both the English Channel and the Atlantic Ocean has no hope? Surely they should just pack up shop, and either adopt OpenStack/Eucalyptus fast… or find a new line of work?

Jeremy Jarvis disagrees, vehemently. And he’s not alone. Look at Edinburgh’s Flexiant, Glasgow’s SymetriQ, and a whole host of other companies that have built their own solutions from nothing. Others, of course, have recognised the value in taking OpenStack, Eucalyptus, Microsoft’s Windows Azure, and similarly established technologies, and making them their own. Look at Eduserv’s Swindon data centre, or the hosted desktops from East Yorkshire’s own GoCloud.

So how can Brightbox (or Flexiant, or SymetriQ, or any of the other non-conformers) compete? How, indeed, can they survive? Jarvis suggests that “Buy British” continues to carry weight here. Even without raising the scary (and often, actually, wholly irrelevant) spectre of PATRIOT Act-powered snooping, a significant proportion of UK (or European) companies like the idea of buying services from UK (or European) suppliers. They like that the documentation is spelled correctly. They like that telephone support is (more or less) in their timezone. They like that the development team shows up at local events, and that it’s a person buying the drinks in the bar, rather than the disembodied marketing budget of some far-off corporation.

Purchasing decisions for something like cloud infrastructure are complicated. Often, they’re probably quite illogical. Price isn’t always the deciding factor (and even if it were, smaller providers like Brightbox aren’t ridiculously expensive in comparison to their larger competitors). Having been the friendly face at a local developer event might swing that big contract. Or having a “nice” API. Or implementing niche features in firewalls, or networking, or port forwarding might each grab the attention — and loyalty — of specific sectors of the long tail. Tesco might sell ‘everything’ to ‘everyone,’ but we still have room in our lives for the SPAR corner shop, and for the upscale deli with the nice cakes.

The same’s true in the cloud, although I can’t help feeling that we’re going to see quite a rapid decline in entirely new cloud infrastructures as the next generation of niche cloud boutiques take OpenStack or Eucalyptus and mould them to their requirements. They probably won’t be making those decisions because (like we Clouderati) they agonise endlessly about interoperability or portability or the de facto standard of the Amazon Web Services stack. For most of their SME customers, those things simply don’t matter. Instead, they’ll be adopting OpenStack or Eucalyptus because the grunt work (and the marketing) has been done. It simply costs less to take something off the shelf than to develop it yourself from scratch. But for companies like Brightbox, where that investment has already been made? Well, for them there may still be plenty of prospective customers out there.

Related articles

• Eucalyptus grabs $30M from IVP to push the open-source private cloud forward (venturebeat.com)
• Announcing Brightbox Cloud – the UK’s first true IaaS platform! (brightbox.co.uk)
• Platform as a service ready to rev up, says Gartner (zdnet.com)
• Hosting providers aim to challenge cloud leaders (zdnet.com)
• Joyent Brings Its Public Cloud To Europe (techweekeurope.co.uk)
• Rackspace launches OpenStack-powered next-gen public cloud (venturebeat.com)
• Ubuntu Cloud Portal: Brightbox 12.04 daily images now available, discounts for testers and Ubuntu members (cloud.ubuntu.com)
• Ubuntu Linux heads to the clouds (zdnet.com)

Image via Wikipedia

The Open Knowledge Foundation (OKFN) promotes the creation, dissemination and use of ‘open knowledge.’ As part of this activity OKFN developed a data repository called CKAN, and has seen this become increasingly important to a range of data dissemination activities such as data.gov.uk and publicdata.eu.

In this podcast I talk with OKFN Director Rufus Pollock and CKAN Product Owner Irina Bolychevsky, to learn more about CKAN, its use in the context of open data, and the wider implications for dissemination of any data (whether open or closed).

Following up on a blog post that I wrote at the start of 2012, this is the tenth in an ongoing series of podcasts with key stakeholders in the emerging category of Data Markets.

Related articles

• Open Knowledge Releases Open Data Handbook 1.0 (readwriteweb.com)
• Data Market Chat: Leigh Dodds discusses Kasabi (cloudofdata.com)
• Data for the public good (radar.oreilly.com)
• Data Market Chat: Stephen O’Grady of RedMonk examines the bigger picture (cloudofdata.com)
• Data Market Chat: Nick Edouard discusses BuzzData (cloudofdata.com)

CEO Shion Deysarkar talks about the development of the web crawling capability within his other company, 80legs, and then goes on to explore the ways in which this impressive capability is being put to work in locating authoritative and comprehensive data for which customers are prepared to pay.

Corroborating different sets of data from across the web, Deysarkar argues, results in a resource more accurate, more timely, and more complete than is available elsewhere.

Datafiniti is currently in beta, and initially focussed upon providing data to address the needs of specific industry verticals.

Following up on a blog post that I wrote at the start of 2012, this is the seventh in an ongoing series of podcasts with key stakeholders in the emerging category of Data Markets.

Related articles

• One API for the Web, Oh the Possibilities (programmableweb.com)
• Datafiniti: Is a Successful Data Search Engine Possible? (arnoldit.com)
• Data Market Chat: the podcasts are a-coming…(cloudofdata.com)
• Data Market Chat: Tyler Bell discusses Factual (cloudofdata.com)
• Data Market Chat: Hjálmar Gíslason discusses DataMarket.com (cloudofdata.com)
• Data Market Chat: Chris Hathaway discusses AggData (cloudofdata.com)
• Data Market Chat: Flip Kromer discusses Infochimps (cloudofdata.com)
• Data Market Chat: Nick Edouard discusses BuzzData (cloudofdata.com)
• Data Market Chat: Stephen O’Grady of RedMonk examines the bigger picture (cloudofdata.com)

Image via CrunchBase

RedMonk don’t offer a data marketplace and, so far as I know, they have no intention of doing so. Nevertheless, this series of data market podcasts would not have been complete without an opportunity to hear what RedMonk co-founder Stephen O’Grady had to say. A blog post of his, from late last year, was the thing that finally persuaded me to stop thinking about a series on data markets and actually get on and do it.

So blame Stephen. Or thank him. Depending upon how you feel about this series so far.

Drawing upon some thinking about markets, and a lot of experience from the development of sustainable models for open source, Stephen packs a lot into this half hour conversation. If you’ve been wondering about the space, or trying to understand whether or not its viable, this should be compulsory listening.

Following up on a blog post that I wrote at the start of 2012, this is the sixth in an ongoing series of podcasts with key stakeholders in the emerging category of Data Markets.

Related articles

• Nurturing the market for Data Markets (cloudofdata.com)
• Data Market Chat: Tyler Bell discusses Factual (cloudofdata.com)
• Data Market Chat: Chris Hathaway discusses AggData (cloudofdata.com)
• Data Market Chat: Hjálmar Gíslason discusses DataMarket.com (cloudofdata.com)
• Data Market Chat: Flip Kromer discusses Infochimps (cloudofdata.com)
• Data Market Chat: Nick Edouard discusses BuzzData (cloudofdata.com)

Image via Wikipedia

Openness is undeniably cool right now, at least if you move in the slightly odd circles that I do. Openly available scientific papers are disrupting the world of scholarly publishing (which may not be all good, but that’s a post for another day). Openly available university courses are finally beginning to work out how to offer meaningful accreditation to students. Openly accessible data from government agencies around the world bulks out almost every data marketplace, and anchors many an analysis. Openly available code for cloud infrastructure or networking is challenging the hold of the tech world’s giants. Everywhere you look, ‘incumbents’ are apparently being ‘challenged’ and ‘disrupted’ by the power of open.

The truth, of course, is a little more complex and a lot more nuanced, as business models shift and evolve just like they always have. In sustainable systems, some people still need to be rewarded (often through being paid) for their effort. And in sustainable systems, paying someone can often be a pretty straightforward means of ensuring that you have a throat to choke if something breaks; big companies adopting open source often seek a proper financial relationship with someone who installs and maintains the ‘free’ software or hardware they’re depending upon.

One area of openness that I’ve been involved with for about ten years is that of open licensing for both creative works and data. And it’s come a very long way.

Here in Europe, for example, the (badly flawed) 2003 Public Sector Information Directive is under review, and there’s every likelihood that the replacement will make a number of sensible moves toward greater openness, transparency, and reusability for publicly funded data. As the EPSI Platform site notes today, Andrés Nin proposes going a step further than the European Commission is currently contemplating, by instituting a common open license across Europe;

“The creation of a single public information re-use space in Europe requires much more, it requires a common European OpenData license applicable to all data generated by European public administrations.”

I would certainly welcome a model license that European member states might be enabled to use. I’d also welcome — and support — vigorous efforts to dissuade individual member states or ministries from their usual practice of tweaking and otherwise modifying perfectly good documents in order to demonstrate how ‘special’ or ‘different’ their circumstances apparently are. When will they all realise that they are neither as special nor as different as they like to think?

But — and it’s a big but — it seems unwise, premature, and unhelpful to even begin to suggest that such a license might be mandated across Europe. It isn’t required, and attempts to develop a single document that everyone could accept would be an unhelpful distraction that would result in something so bureaucratic, so ringed in opt-outs and prevarications, as to be utterly worthless. It would also, in all likelihood, be one of those exercises in which the process very quickly subsumed the point. A prime candidate for, in the words of an old boss, being too busy to be effective.

Related articles

• Branding academic publishers ‘enemies of science’ is offensive and wrong (guardian.co.uk)
• Open Data: Europe Starts to Get It (opendotdotdot.blogspot.com)
• Open Data in Europe gets a huge boost from new EU rules (thenextweb.com)

Image via Wikipedia

Last week, open standards body OASIS unveiled yet another shiny new standards effort. The OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) Technical Committee hopes to make it “easier to deploy cloud applications without vendor lock-in,” and to support moving from one cloud to another. The usual suspects — the likes of IBM, CA, and Cisco — are on board. The usual holdouts — Google and Amazon, of course — are not. So what is TOSCA trying to achieve? How does it fit alongside all the dead, dying, or ponderously deliberating cloud standardisation efforts that have gone before? And without the giants of the cloud, is there really any point bothering?

As I’ve probably mentioned before, involvement in various national and international standardisation efforts played a big part in my early career. I went to the working group meetings in odd (but often beautiful) locations. I participated in the conference calls. I engaged on the mailing lists. I drafted and edited and reviewed the documents. I completely buy into the idea that there is a place for foundational standards, developed through consensus-building and maintained for the long haul by organisations that stand apart from the vested interests and their competing agendas.

I also believe that there’s a time and a place for these standardisation efforts. Do it too soon, and we end up ossifying something that needs to be in a state of flux. When you don’t know what the best way to prepare a meal is, it’s too soon to print the recipe book. We need to try different approaches, and we need to be able to throw away the attempts that didn’t work out. More worryingly, standardisation efforts can be used for political ends. They can be little more than a rod with which to beat the (usually dominant) competition. At best a distraction, or a talking shop for those unwilling or unable to just get on and do something. At worst, one amongst a toolchest of dirty tricks in a broader war for hearts, minds, and — ultimately — wallets.

The cloud market is a fascinating place. There are leaders and there are followers. There is innovation, and there is competition. There is agreement, and there is debate. For all the rhetoric, and all the posturing, we really don’t yet know the right answer to many of the cloud’s questions.

Maybe TOSCA and the Open Data Center Alliance and IEEE and the rest are — still — too early, and should be content to let the market thrash out a few more of these issues before anyone tries to write anything down? And when it is time to write some stuff down, let’s make sure we focus on specific, finite, tangible, atomic tasks rather than “the cloud.” As Dave Roberts commented in regard to TOSCA’s scope;

“That goal is so large, that I think it’s probably unbounded. When problems get unbounded, the best you can ever hope to achieve is to solve a large enough subset of the problem that the solution is still interesting. If you can’t achieve that, people ignore the solution because it fundamentally doesn’t help them. There is always an ‘interesting’ part of the problem space that they have to solve a different way, and that undercuts the use of the partial ‘solution.’”

And as for Tosca? Things didn’t end well for her, did they? Might TOSCA’s fate, too, be sealed?

Related articles

• Tech giants back standard for cloud portability – InfoWorld (infoworld.com)
• Proposed spec aims to nix cloud lock-in (gigaom.com)
• On TOSCA and Cloud Standards. MyPOV (diversity.net.nz)

I’ve been a fan of Om Malik‘s boutique analyst site, GigaOM Pro, pretty much from the outset, and happily renew my subscription each year. The site covers a wide range of industry topics, and those Quarterly Wrap-ups are worth the fee all by themselves.

I’ve written a few reports for them in the past, but was delighted when Mike Wolf got in touch to see if I fancied trying my hand at curation on their Infrastructure/Cloud channel.

So next week (from 31 January) I’m going to be gathering and commenting upon links from around the web, writing a daily ‘Today in Infrastructure,’ and finishing off with a Weekly Update. If you’re not (yet!) a subscriber, why not sign up for a free seven day trial and join me for the start of my little adventure?

If there’s something you think I should be covering, do let me know.

Related articles

• Free GigaOM Pro Webinar: The Scalable Cloud (gigaom.com)
• GigaOm raises $2.5M, claims 10,000 Pro subscribers (venturebeat.com)
• The Cloud has a place, even inside heavily regulated industries (cloudofdata.com)
• Talking Scalable Clouds with GigaOM Pro and Limelight Networks (cloudofdata.com)
• GigaOM Structure 2011 (datacenterknowledge.com)
• Join GigaOM at Big Data on March 23 in New York City (gigaom.com)

Towards the end of George Orwell’s allegorical take on the Stalinist Revolution, the pigs of Animal Farm take on the trappings of the humans they supplanted, shifting ideologically from ‘Four Legs Good, Two Legs Bad’ to declare ‘Four Legs Good, Two Legs Better!’ as they rise to stand on their hind legs.

The pigs’ dogmatism forces a series of increasingly convoluted rationalisations, until they end up professing exactly the opposite of their original position. Black, it seems, really can be white… but there’s absolutely no room for grey.

With data, current moves toward ‘open’ are certainly to be lauded, and we should continue to demonstrate the benefits of more equitable access in persuading those who have yet to realise the opportunities for rethinking their business.

However, I’ve been in too many situations recently where persuasion, encouragement and demonstration have been cast aside in favour of brow beating, castigation and vitriol. Anyone who fails to immediately throw open the doors to their data vaults is, the argument increasingly seems to go, cruelly, wantonly, and entirely unreasonably standing in the path of truth, justice, and the {insert name of country} way. The language is intemperate, and the unspoken undercurrent of feeling seems almost to lump these evil data hoarders with the most vile underminers of social cohesion.

Nonsense.

Open Data is a good thing, and we could benefit from an awful lot more of it. But the arguments surely shouldn’t be religious (‘Open’ is better than ‘Closed’) or so polarised that compromise typically becomes impossible. Instead, we need collectively to demonstrate the value of change, and we need to understand and respect the positions of the market’s incumbents. Current practice should never be accepted as an excuse for lack of change, but all too often it may actually mask quite a good set of reasons.

Where data are currently sold, can we (as was to some extent done for the Ordnance Survey) calculate the costs of data collection, curation and sale, and demonstrate convincingly that more money could be made by removing that initial barrier to access?

Where a data holder participates in an existing data sharing arrangement with their peers, surely we can gather the evidence to demonstrate the likely effect of opening parts of that value chain… without destabilising an otherwise useful set of collaborations?

Where large quantities of low value data (such as a customer’s address) are stored and managed alongside highly valuable business data (the facts of a customer relationship), we can certainly set about demonstrating the ways in which a more open approach could pay dividends; instead of managing that postcode yourself, share a little in order to benefit from the work done by others on tracking past, current, and future changes of address.

Then again, maybe we should just scream and swear at all those data-hoarding dinosaurs, without trying to understand them or engage with their fears, concerns, and counter-arguments. It’s much easier that way.

Four Legs Good. Two Legs Often Quite Good, Too!

Related articles

• The black market for data (radar.oreilly.com)
• GTEC 2010: David Eaves on Open Data: ‘Just Do It!’ (techvibes.com)
• The Linked Open Data and Pavlova (blogs.talis.com)
• Nottingham University offers masterclasses in dealing with open data – for free of course (guardian.co.uk)
• Talking with Richard Stirling about progress with data.gov.uk (cloudofdata.com)
• ICO launches consultation on data sharing code of practice (v3.co.uk)
• How Open Data is Used Against the Poor (readwriteweb.com)

The OpenStack activity was unveiled by Rackspace, NASA, and their partners back in July, and is on track to deliver functional initial releases in the next few weeks. We discuss the relationship between OpenStack’s deliverables and earlier developments from Rackspace and NASA’s Nebula project, and begin to explore the implications of an Open Source Cloud Computing stack for the wider industry.

This podcast was recorded on Friday 10 September, 2010.

During our conversation we referred to the following resources;

• Amazon
• Amazon EC2
• Amazon S3
• Apache license
• Citrix
• Cloud.com
• Dell
• GoGrid
• HP (and my podcast with Russ Daniels)
• IBM (and my podcasts with Kristof Kloeckner and Ric Telford)
• KVM
• Linux
• Microsoft (and my podcast with Amitabh Srivastava)
• mySQL
• NASA
• NASA Nebula (and my article for GigaOM Pro)
• OpenStack
• OpenStack code launchpad
• OpenStack design conference, November 2010
• Oracle
• Rackspace (and my podcast with Lew Moorman)
• Rackspace CloudFiles
• VirtualBox
• Werner Vogels, Amazon CTO, discussing ‘false Clouds’ at GigaOM’s Structure conference
• XenServer

Related articles by Zemanta

• OpenStack sets some Texas-sized deadlines (zdnet.com)
• OpenStack: Rackspace and NASA Nebula Join Forces for Open Cloud Ecosystem (readwriteweb.com)
• NASA gives OpenStack instant credibility (zdnet.com)
• Impact of OpenStack Project Goes Beyond the Cloud Industry Leaders (readwriteweb.com)
• Space IT, the final frontier (radar.oreilly.com)
• What’s next for OpenStack’s cloud efforts (news.cnet.com)
• Rackspace Announces Open Source Cloud Platform (informationweek.com)

When I was a child, I wanted to be a spaceman. No great surprise there, perhaps, and also no great surprise that I — like so many others — never got to achieve that dream.

Still, when the opportunity presented itself to write a space-y piece as my latest contribution over on GigaOM Pro, I jumped at it.

It was fascinating to hear about the very different efforts at the European Space Agency and NASA to harness Cloud Computing… and it brought back some of that childhood excitement in the process.

Thanks to William O’Mullane at ESA and Chris Kemp at NASA Ames for their time and insight.

And if a slot opens up on the next non-robot mission, you know where to reach me…

Related articles by Zemanta

• NASA’s Nebula: The Cloud in a Container (datacenterknowledge.com)
• To Space and Beyond: The Rise of Research-driven Cloud Computing (gigaom.com)
• Cassini Shows Saturnian Roller Derby, Strange Weather (spacefellowship.com)
• Ames expects Obama budget will mean growth (insidehpc.com)