Paul Miller - The Cloud of Data » Cloud computing

CloudCamp London: the Big Data Special

Paul Miller — Wed, 25 Jan 2012 21:59:14 +0000

Image by Kevin Krejci via Flickr

The CloudCamp unconference returned to London for the 14th time this evening, regaling a capacity crowd in the Crypt below Clerkenwell’s St James Church with several hours of discussion and debate on the somewhat elusive topic of ‘Big Data’.

Rather rough notes of the proceedings follow, after the break.

LEF‘s Simon Wardley kicked proceedings off as usual, once again managing to pepper an on-topic canter through the topic with a seemingly never-ending stream of Flickr images of cats… and analogies to electricity. You possibly had to be there? His core message, though? There’s nothing new under the sun… and the cycles of change just keep on coming.

Next, Peter Matthews from CA Labs, on “is big data mutually compatible with the cloud?” Erm, yes. Data volumes with big data are so large that it’s difficult to move it around… which creates opportunities for lock-in that vendors may wish to seize. And then he was out of time.

Next, Fujitsu’s Mark Wilson on ‘Structuring Big Data.’ He’s actually talking about Linked Data, a topic I’ve dug into before here and over on semanticweb.com – Linked Data could be/ might be the effective realisation of the decade-old Semantic Web dream. Big Data means masses of unstructured or semi-structured content, presenting a management headache of previously unanticipated proportions. Linked Data, he argues, creates the mechanism to link all of this data together from across disparate sources. Yes, but it’s easier to say than to do… And in 5 minutes he really couldn’t explain enough to persuade the audience. Linked Data should be “the optimal reference source,” he said. It should be “a broker for all data sources,” and we should “think about integration, not duplication.” Yeeeeees… But.

Next, Canonical’s Nick Barcet, talking around scalability, Ubuntu, package management, configuration management, etc. Not wholly sure what the point was, I’m afraid.

Next, Chris Swan from UBS – big data and security. “If you’ve got security controls that aren’t properly monitored, then they don’t matter.”

Next, Tom Leyden of Amplidata – Big “Unstructured” Data in the Cloud. Data storage to increase 30x over the next decade, but staff will only increase 50% over the same period. Challenge in the 90s, as existing storage and analysis technologies struggled to cope with new data volumes. Seeing similar problems today with data streaming from sensor web, etc. Traditional file systems cannot cope. Object Storage the way forward ?

Next, Alex Farquhar – “Cloud v Big Data.” Not really versus… but intersection of the two. Too much discussion of his company, Forward. Just talking about how his company uses cloud to provision IT resources. Might work as a conference presentation or case study – not sure it fits as a 5 minute lightning chat. Around 60TB of data at Forward. Diverse and vital. Using Hadoop cluster – 24 nodes on-premise. Rationale (proximity to the cluster) seemed odd. That can be true, but not clear that it really needs to be the case here?

Next, Alaric Snell-Pym, on Scaling Hadoop. Trying to overcome Hadoop’s I/O bottleneck. Explaining basics of Hadoop and Map/Reduce – no one else has. Explains use of HDFS and ‘selective reading’ to manage lots of small tables and overcome the problems of I/O.

Next, Matt Wood from Amazon. Talking about genetics and the human genome. It’s an analogy. Human Genome Project took years and millions of dollars. Development of gene sequencing machines led to a step change – dramatic drop in cost of sequencing DNA. Like the cloud, anyone? But… the machines create an analysis challenge, because they generate so much data. Cloud offers “collection of productivity tools” to help scientists work with this data collaboratively and (relatively) affordably. A perfect example of a lightning presentation, unlike most of those who preceded him.

And finally, an impromptu slot from HP’s Joe Weinman. A quick overview of current thinking behind his latest book. This one could have gone for much longer… Good stuff.

And that’s the lightning talks finished. Now, the panel, and Simon Wardley’s search for “experts” and “volunteers.”

…and unfortunately, your scribe was ‘volunteered’ as an ‘expert’ by Mr Wardley… and here end the notes. It was great to have Amazon’s Werner Vogels sneak in, and lob comments into the panel, though…

Great event, though with the usual mix of people you wish could have talked for longer… and people you wish wouldn’t have spoken.

The brave new world of big data & Hadoop (venturebeat.com)
Big VCs Invest In Big Data Startup Continuuity (techcrunch.com)

TOSCA may prove a prescient name for new cloud standards effort

Paul Miller — Tue, 24 Jan 2012 21:00:23 +0000

Image via Wikipedia

Last week, open standards body OASIS unveiled yet another shiny new standards effort. The OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) Technical Committee hopes to make it “easier to deploy cloud applications without vendor lock-in,” and to support moving from one cloud to another. The usual suspects — the likes of IBM, CA, and Cisco — are on board. The usual holdouts — Google and Amazon, of course — are not. So what is TOSCA trying to achieve? How does it fit alongside all the dead, dying, or ponderously deliberating cloud standardisation efforts that have gone before? And without the giants of the cloud, is there really any point bothering?

As I’ve probably mentioned before, involvement in various national and international standardisation efforts played a big part in my early career. I went to the working group meetings in odd (but often beautiful) locations. I participated in the conference calls. I engaged on the mailing lists. I drafted and edited and reviewed the documents. I completely buy into the idea that there is a place for foundational standards, developed through consensus-building and maintained for the long haul by organisations that stand apart from the vested interests and their competing agendas.

I also believe that there’s a time and a place for these standardisation efforts. Do it too soon, and we end up ossifying something that needs to be in a state of flux. When you don’t know what the best way to prepare a meal is, it’s too soon to print the recipe book. We need to try different approaches, and we need to be able to throw away the attempts that didn’t work out. More worryingly, standardisation efforts can be used for political ends. They can be little more than a rod with which to beat the (usually dominant) competition. At best a distraction, or a talking shop for those unwilling or unable to just get on and do something. At worst, one amongst a toolchest of dirty tricks in a broader war for hearts, minds, and — ultimately — wallets.

The cloud market is a fascinating place. There are leaders and there are followers. There is innovation, and there is competition. There is agreement, and there is debate. For all the rhetoric, and all the posturing, we really don’t yet know the right answer to many of the cloud’s questions.

Maybe TOSCA and the Open Data Center Alliance and IEEE and the rest are — still — too early, and should be content to let the market thrash out a few more of these issues before anyone tries to write anything down? And when it is time to write some stuff down, let’s make sure we focus on specific, finite, tangible, atomic tasks rather than “the cloud.” As Dave Roberts commented in regard to TOSCA’s scope;

“That goal is so large, that I think it’s probably unbounded. When problems get unbounded, the best you can ever hope to achieve is to solve a large enough subset of the problem that the solution is still interesting. If you can’t achieve that, people ignore the solution because it fundamentally doesn’t help them. There is always an ‘interesting’ part of the problem space that they have to solve a different way, and that undercuts the use of the partial ‘solution.’”

And as for Tosca? Things didn’t end well for her, did they? Might TOSCA’s fate, too, be sealed?

Tech giants back standard for cloud portability – InfoWorld (infoworld.com)
Proposed spec aims to nix cloud lock-in (gigaom.com)
On TOSCA and Cloud Standards. MyPOV (diversity.net.nz)

Nurturing the market for Data Markets

Paul Miller — Mon, 09 Jan 2012 20:20:32 +0000

From Microsoft’s Azure Data Marketplace to the eponymous DataMarket, or InfoChimps, Factual, and Kasabi, there’s resurgent interest in the venerable business of collecting, curating, and commercialising data created by others. But despite investment and innovation, there isn’t yet the matching evidence for much use or — even — interest amongst prospective customers. In principle, at least, these data markets should be providing valid, viable, and valuable services to a market that is potentially enormous. So why aren’t more users rushing to get at these sites?

In many ways, the core concept of the data marketplace is nothing new. Companies like Bloomberg, Nielsen and Experian have built (extremely) profitable businesses by aggregating data, quality checking it, and selling it on. Often their customers could have gone directly to the source(s) and paid far less, but they don’t. The convenience and quality assurance of dealing with a single — reputable — source is perceived to have value. A brand like Bloomberg’s is associated with trustworthiness and authority, and the brand of the marketplace is far more prominent than the data sets upon which it is built.

Similar sites have also served the needs of those seeking data for free, with IBM’s ManyEyes project, Freebase (acquired by Google), Hans Rosling’s Gapminder or The Guardian‘s Data Store amongst those typically mentioned. Current government enthusiasm for ‘transparency’ has fed all of these sites with data, and led to creation of large government-specific data repositories such as data.gov.uk.

The commercial services like Bloomberg have tended to focus upon specific domains (finance, in Bloomberg’s case) or types of data. They have also tended to be eye-wateringly expensive; aimed squarely at the small market segment for whom the data are mission-critical and the fees are affordable. The free services like Gapminder also tend to focus (global development statistics in this case). Other, perhaps, than experiments like ManyEyes, both the free and the commercial sites tended to aim for a degree of comprehensiveness and authority. They wanted to become the place to turn for their type of data.

But for the new generation of data markets, the picture becomes far less clear. They tend to be catholic in their data acquisition policies, they typically don’t even attempt comprehensiveness, they mix free (almost all of them hold identical large swathes of government data from the US, the UK, and elsewhere) with commercial data, and they continue to feel their way toward business models that might prove sustainable for the long haul. Perhaps more seriously, they appear almost schizophrenic with respect to brand projection, attempting to push both their own brand and those of the data sets they host in ways that can confuse far more often than they enlighten.

In attempting to differentiate themselves, today’s data markets are seeking to add features and functionality in order to be seen as far more than simply places to buy third-party data. They want to become recognised for quality assurance, for data enrichment, or for tools and capabilities that make working with the data easier or more powerful. They want to become sticky, and they want to be seen as different from their competitors. The trick, though, is to explain those features and those differences in ways that make sense to potential customers. Those customers will ultimately pay for functionality and utility, not for gimmicks or under-the-hood technological distinctions that have no real impact upon getting on with the job in hand. Are today’s data markets describing their features in ways that help prospective customers to understand why they should be chosen over the alternatives? Not really. At least, not yet.

Also, as RedMonk’s Stephen O’Grady touched upon amongst a set of related issues, we’ve really not begun to see much evidence of price competition. There are too few suppliers, each with their fiercely loyal bands of tame users (‘customers’), and too few people prepared to shop around for the best deal.

The new data markets are still young. Understandably, they are still feeling their way in order to understand what the market wants, how much it is prepared to pay for what it wants, how large the market might be, and what their individual niche within that broader market might look like. Earlier models, based upon almost monopolistic domination of specific verticals and polarised pricing, offer some lessons but are ultimately unsatisfactory blueprints for this more competitive, open, and complex environment. Beyond specific domains like finance (which may be ripe for disruption), the data markets must struggle to convince prospective customers that they have something of value to offer. Those customers may already have their own processes for obtaining data. They may generate the data themselves, or expect — as so many do — to be able to access what they need for free. They are perhaps suspicious of data produced by third parties who are, in other contexts, their competitors, and they are almost certainly unwilling to allow ‘the competition’ to benefit from their own data. They invariably do not understand the costs associated with gathering and quality-assuring data, or the challenge of preparing different data sets in order that they may meaningfully be combined. And into this, the fledgling data markets must insert themselves, market themselves, and sell themselves. They must change behaviours, they must challenge presumptions, they must alter working practices, and they must persuade their new customers that all of this pain is worth paying for. A tall order, indeed, but necessary if any of them are to realise their potential.

The European Commission, at least, begins to comprehend the scale of the challenge. A set of projects are currently being finalised, and this year will see European SMEs given the funding to boot-strap a number of new data sources. With Commission funding, it is hoped, the chosen projects will be able to explore models by which data can be created, curated, shared and re-used in a manner that is cost-effective and ultimately sustainable. The funding should enable these projects to reach viable scale, and give participants the freedom to explore alternative commercial models. The projects will be announced shortly, but only time will tell if the funding and the incentives are sufficient to break through the barriers that prevented any of these markets from forming by themselves.

But outside the rather artificial bubble created by European public funding, there is a lot of work to do. Investors are intrigued by — but still wary of — the opportunity. Infochimps is spending its way through over $1.5 million of investment, Factual has almost $30 million, and companies like Talis and Microsoft are making not-insignificant investments in their own efforts. We’re all still experimenting, but with the real market for these services currently falling far short of the money at stake, it mustn’t be long before investors start asking harder questions. Back in 2010, Pete Soderling and Pete Forde described data as a $100 billion market. The data markets may be after a significant chunk of that but, today, they’re not even close.

The ways that data markets are attempting to differentiate themselves, and the work being done to understand the market opportunity here, will have to wait for subsequent posts.

Disclosures: I am a former employee of and current shareholder in Kasabi’s parent company, Talis. The European Commission is, from time to time, a client.

With Factual, 1 API now unlocks data for 55 million places (gigaom.com)
‘Roswell’: Another key component of Microsoft’s cloud strategy (zdnet.com)
Infochimps Acquires Y Combinator Startup Data Marketplace, Expanding Brand Holdings and Online Presence (prweb.com)

The myth of a data free trade policy

Paul Miller — Thu, 05 Jan 2012 14:59:39 +0000

The border between the USA and Canada, in Washington State

In my last post I looked at the USA PATRIOT Act, and at some of the ways in which it exemplifies differences in attitude and approach on either side of the Atlantic. In our increasingly connected world, these differences begin to pose quite serious challenges for those wishing to join up, to aggregate, and to operate at scale. In this post I’ll take a look at one particular case in which matters must soon come to a head; the current enthusiasm for cross-border data flows, and what GigaOM’s Derrick Harris refers to as “free trade for data.”

At first glance, the concept of free trade for data makes a lot of sense. Of course data should be able to move with reasonable freedom from country to country. As someone who once had the job title of ‘Interoperability Focus’ I’m bound to agree that international standards should normally be used to promote interoperability and transparency. Few, surely, would not welcome cross-border arrangements to encourage entrepreneurial reuse of data, or to ensure that people in different countries can access popular online services headquartered overseas?

Unfortunately, the bulk of the evangelism around reaching new agreements here is nationalistic, partisan, and closely tied to particular world views. From here in Europe, American posturing on the topic grates. From America, we Europeans no doubt appear protectionist and over-cautious. And elsewhere in the world, governments and companies with valid contributions to make cry out to be heard amidst the trans-Atlantic babel.

Derrick’s post from last November is a case in point, discussing submissions by US companies (Visa, Mastercard, Microsoft, Oracle, Salesforce, etc) to a US body (the National Foreign Trade Council), in an attempt to influence US government policy with respect to its peers around the world. The topic was Cross-Border data sharing, but similar companies from overseas were not involved. Nokia? Vodafone? Baidu? SAP? HSBC? Nope, nope, nope, nope, nope. Companies embracing a particular world view, rooted in a particular culture, come together to draw up recommendations that (probably in good faith) reflect that world view and those cultural norms. Whether implicitly or explicitly, those recommendations then seek to project that world view onto other countries, other cultures. We all do it. We all bring our baggage, our beliefs, our presumptions. Sometimes we know when we’re doing it, and we can either carry on regardless, or we can attempt to account for alternative approaches. But all too often the cultural norms are so ingrained that we forget they’re there. We assume that they’re normal. We assume that they’re shared. And then we’re surprised when Australians balk at receiving the ‘summer’ release of software in June, when Americans don’t consider that piece of personal data to be sensitive, or when Europeans think Government really should be involved in regulating a social networking site.

If we want to ensure the unimpeded flow of data across borders — and we should — then we need to begin by recognising that the places and people on either side of that border are very likely to be different. Their attitudes are different. Their needs are different. Their aspirations are different. Their laws are different. In the early days of the Internet and the web, legislation, policy and even expectation did not really exist. Almost by default, US attitudes and presumptions tended to apply unless a particular country cared enough to institute something different inside their own borders. As we become more and more interested in territoriality and jurisdiction with respect to data, that naive innocence no longer applies. There is no longer a blank canvas upon which the innovators can paint their hopes. We have policies, regulations, and laws. We have populations that have experienced today’s web, and we have a media quick to interject its perspective.

Today, far more than at the web’s birth, we have to engage in public dialogue about what we want to achieve, and why. We cannot achieve Derrick’s aspiration of free trade for data and leave every local law, policy and procedure untouched. But nor can we achieve it by projecting a single world view around the globe, sweeping all of those prior laws aside.

Rather than entrench behind a US ‘position,’ a European ‘position’ (including 24 variant positions, two abstensions and an opt-out), a Chinese ‘position,’ and so on, can’t we begin to understand what some of the real opportunities — and concerns — might be?

Big companies that operate internationally (like those developing the document (PDF) Derrick discussed) absolutely need to come together, to share their perspectives on the pain of moving data around. But for those companies only to be American is insane, and counter-productive. Nokia has perspectives to share here, as do HSBC or Vodafone. Let’s hear them in the same forum. Let’s also hear individual governments, speaking up for the concerns and desires of their citizens. Let’s hear the citizens themselves, when they care enough to express an opinion. But let’s hear all of it early, before it becomes entrenched in a set of contradictory official statements.

Then we might arrive at a sensible approach to ensuring free trade for data, rather than the projection of an American ideal upon the rest of us.

And that sounds like a good topic for one of those panels at Davos later this month…

Advancing the free flow of information (googlepublicpolicy.blogspot.com)
PATRIOT Act clouds picture for tech – POLITICO.com Print View (mbcalyn.com)
Border accord (economist.com)
ANALYSIS: What the new border deal means for you (cbc.ca)

Microsoft, the USA PATRIOT Act, and European cloud computing

Paul Miller — Thu, 05 Jan 2012 13:14:09 +0000

Microsoft announced last month that its Software as a Service (SaaS) offering, Office 365, will better comply with European guidelines to ensure that customer data is adequately protected. This move is certainly welcome, but the long-armed spectre of the USA PATRIOT Act continues to hang over Microsoft and other US companies, regardless of customers’ nationality or the country within which Microsoft might physically host a particular customer’s data.

The PATRIOT Act’s acronymic name may evoke harmless images of bunting, parades, and national anthems, but the reality is rather different. A product of America’s post-9/11 entrenchment, the Uniting (and) Strengthening America (by) Providing Appropriate Tools Required (to) Intercept (and) Obstruct Terrorism Act of 2001 affords the Federal Government wide-ranging and far-reaching powers that show little — if any — respect for geographic boundaries or inconveniently contradictory local legislation. A US company (like Microsoft or Amazon) is subject to the Act’s powers all around the world. A US citizen’s data, stored in a US company’s data centre that is physically situated in the United States is subject to the Act, and everyone might be reasonably comfortable with that. But so is a German citizen’s data, stored in an Amazon data centre in Ireland; and German, Irish and European lawmakers appear almost powerless to intercede.

European countries tend to be stricter about use (and abuse) of personally identifiable information than the US. Although surveys identify some national differences, it also appears that Europeans broadly embrace the approach taken by their governments. And, anecdotally, conversations with European and American entrepreneurs and European and American individuals repeatedly point to rather different sets of basic presumptions operating on either side of the Atlantic. Europe’s Data Protection Directive, and its implementation in national legislation such as the UK’s Data Protection Act, are clear about the ways in which a citizen can expect data about themselves to be collected, stored, shared and used. The penalties for intentional abuse could probably be tougher, but the sentiment remains clear. The Safe Harbor Principles provide mechanisms by which US companies can self-certify that their normal operating procedures meet European standards (Microsoft, Google, Amazon and many others do this). The February 2010 ‘model clauses‘ that Microsoft embraced last month codify some of these protections in a manner that — theoretically — makes it easier for customers’ lawyers to understand what Microsoft will do with their data. It’s unlikely that the legalese (PDF) will actually make things any clearer for the average customer, though.

So, from the perspective of Europe’s governments and citizens, and for US companies that choose to trade here, things appear more or less ok. Personally identifiable data can be collected, stored, shared and used, but only within a set of constraints that Europeans broadly seem comfortable with. Unfortunately, all those Safe Harbor self certifications and model clause endorsements are summarily ignored whenever the PATRIOT Act is invoked. Data Protection Directive requirements not to transfer data to random third parties are trumped by PATRIOT Act powers enabling the US Federal Government to take what it wants. Data Protection Directive stipulations that citizens be informed when their data are taken are over-ruled by the PATRIOT Act’s cloak of secrecy. And on and on the list of contradictions continues. And the PATRIOT Act wins every time, because its powers, its penalties, and its backers are so much scarier than the officials in Brussels. Despite tougher language, it’s not clear that sweeping changes to Europe’s data protection directive will really resolve the contradictions. Indeed, once enshrined in law, the proposals will most likely result in more polarisation, not less.

In Europe too, of course, there are exemptions to the data protection legislation specifically intended to permit reasonable use of data by law enforcement agencies and others. This makes sense, and it could be argued that the PATRIOT Act is simply more of the same. But it’s not, because European law enforcement agencies must demonstrate a far clearer need before they’re allowed to — legally — start rooting through a citizen’s data.

It is unlikely that the PATRIOT Act is routinely invoked, or that US officials spend much time reading Europeans’ email. The cloud — even the parts run by US companies — remains broadly safe, secure, and reliable. Safe Harbor provisions, model clauses, and the ability to insist that data normally resides in one territory or another remain an effective means of ensuring that day-to-day cloud operations meet user needs whilst complying with relevant local, regional and international legislation. But, every now and again, the PATRIOT Act will be invoked, and data will be taken. Whilst it’s something to be aware of, it’s probably not something for most people to lose too much sleep over. You’re more likely to lose data yourself, or have it escape into the wild because of an error in your own systems or a malicious hack by a competitor. And you could and would be held accountable for those breaches, in a way that you almost certainly wouldn’t for a PATRIOT Act data seizure.

So the PATRIOT Act may not be as scary as it might now appear. But it remains a visible illustration of a rather more worrying issue; a belief that the laws of one country should be able to trample over the laws of other countries at will — even inside those countries. Further, it suggests a (growing?) disconnect between the attitudes and expectations on either side of the Atlantic. And one particular aspect of that is the subject for my next post.

Image by Flickr user John Morgan

Microsoft’s European ‘cloud pact’ still does not protect data against FISA, Patriot Act (zdnet.com)
E.U. Regulations Become Microsoft Cloud Selling Point (blogs.wsj.com)
Microsoft Pushes Back on EU Cloud Concerns as European Rivals Move In (wired.com)
Defense giant ditches Microsoft’s cloud citing Patriot Act fears (zdnet.com)
PATRIOT Act and privacy laws take a bite out of US cloud business (arstechnica.com)
Microsoft boosts Office 365 regulatory compliance (infoworld.com)
Updated European law will close Patriot Act data access loophole (zdnet.com)
Exclusive: European data protection law proposals revealed (zdnet.com)

Of Kindles and Business Models and Stuff

Paul Miller — Mon, 05 Sep 2011 19:40:59 +0000

Cover via Amazon

Over at TechCrunch, MG Siegler’s 2 September post on Amazon’s new Kindle has generated quite a storm. All across the web, media, bloggers, pundits, analysts and the rest are pointing to MG’s post, getting terribly excited about a new tablet that might actually challenge the iPad; something that so many others have patently failed to do. I am a satisfied and long-time iPad user, but would be the first to admit that a little dominance-challenging is a good thing for all of us. It’s good for Apple, it’s good for developers, it’s good for the supply chain… and it’s good for the user. The cheap Android-powered tablet that MG claims to have seen and used sounds interesting, especially given its tight links to Amazon’s content, consumer relationship, and online services.

However, isn’t it just a different take on the iPad’s proven model?

It’s Android instead of iOS. It’s Amazon’s relationship with my credit card rather than iTunes‘. At least in the United States, it’s the Kindle store and Cloud Player and Amazon Instant Video in place of iBooks and iTunes. It’s 1984 instead of Google Voice. It’s also (probably) a 10-ish hour battery life, and a screen that makes all that poolside reading a no-no. It’s a choice, and therefore (probably) good. It should appeal to people who didn’t warm to the iPad, and it should offer a real choice to those in the market for a tablet. I look forward to being able to see one for real.

But I hope that Amazon don’t lose sight of the opportunity presented by the Kindle device that they already have.

As I mentioned at the start, I’ve had an iPad for some time, and it’s now an extremely useful part of my work-day routine. RSS feeds, Evernote, The Economist, Email, Spotify and Sonos, The Financial Times, and more. Lots of consumption, and a bit of light creation, almost every day.

I also have a Kindle, and use it almost daily as well. Both devices have access to the same purchased books, but (and no, I don’t know why) I tend to read fiction on the Kindle, and tend to read work-related stuff on the iPad. Despite being able to read all the same stuff, the Kindle has sufficient value to be worth the £111 it cost me. It’s easier to hold for long reading sessions. Its screen is easier on the eye. Its battery life is wonderful. It’s easier to hold as you board a plane, removing the need to faff and fiddle in bags during the hand luggage-stowing scrum. The e-ink screen works beside the pool, in those train seats where flashes of sunlight keep falling across screens, in the garden, and everywhere else I try to use it. On last month’s holiday to Crete, the iPad (and work) stayed at home. The Kindle came along.

The Kindle is a reasonably nice piece of hardware (although competing devices from Sony and others have, I feel, tended to be better designed and built). The hardware isn’t the best part of the Kindle, though. That accolade must surely be reserved for WhisperSync, and the incredibly straightforward way in which a Kindle device is able to access Amazon’s massive catalogue without fuss, without wires, without delay. Sony and others blew that. Their syncing was (is?) painful, slow, buggy beyond belief, tethered to a computer, and (at least the last time I tried it) linked to a pathetic catalogue of content.

So the Kindle experience offers a good-enough device, at a good-enough price, tied to a best-in-breed buying experience.

It encourages the owner to buy more stuff, which sounds like a good strategy.

This new Kindle that MG has seen links through to the same buying experience, and may even make it easier to keep on buying. But it appears to give up every single one of the other attributes that made the Kindle strong. It’s bigger, it’s more expensive, its battery is less good, and its screen will be as bad (and good) as the iPad’s. To increase the dominance of the Kindle experience — and of Amazon’s e-commerce juggernaut — I’d suggest that Amazon actually needs to do more with what it already has, rather than sacrificing so many strengths to the battle with Apple.

In the United States (it’s not available elsewhere), Amazon’s advertising-subsidised Kindle is already the company’s best seller. The reading experience isn’t interrupted by ads; owners simply see adverts on the screensaver, instead of pictures of dead authors. By agreeing to have these adverts shown on the device whenever it’s in their bag, on their bedside table, or otherwise being completely ignored, buyers shave $25 off the $139 list price.

Amazon should be bolder, and make that price differential greater.

Why not $99 instead of $139? The purchase of a Kindle isn’t a one-off transfer of a device from Amazon to the buyer, and a reciprocal transfer of money from the buyer to Amazon. It’s the beginning of a relationship. It’s the necessary step to take before buying your first Amazon ebook. It’s the initial purchase in an ongoing series of purchases, as the Kindle owner buys book after book after book. The $139 is a barrier, standing between Amazon and that One Click™-powered relationship with my credit card. Amazon should take a hit here, to begin the longer and more lucrative relationship.

Honestly, I’d be surprised if we don’t see the current Kindle offered for $99 before Christmas.

In fact, forget $99; Amazon should just give it away.

Like €99 and £99, $99 is an emotive price. $99 is an awful lot less than $100. To many people, $99 is almost an impulse buy. $100 is ‘expensive.’ Even better than $99, though, is “Free.” I don’t need to buy anything before I can buy and read a paper book. It’s conceivable that the ebook reader, too, should stop being a barrier to entry, and be given away for nothing. Amazon, of course, needs to recoup the cost of developing and manufacturing the Kindle, and it’s almost inconceivable that it’s managed to do this already. So a Kindle has tangible costs associated with it, and certainly doesn’t cost Amazon nothing. Giving up $99 (or, today, $139) of revenue on every single Kindle may be a bit much for Amazon to swallow, even with the intangible promise of that nebulous ongoing buying relationship. So why not make the promise tangible, and the relationship less nebulous? Take a leaf from the mail order book clubs of yesteryear (do they still exist?), which tied members into a minimum number of purchases each year. “Give away” a Kindle in exchange for a minimum number of book purchases (or a minimum spend with Amazon?) each year. The margins on the purchased books wouldn’t even need — directly — to be enough to cover the ‘lost’ $139. The recipient of the free Kindle would have their relationship to Amazon strengthened in all sorts of ways, making them more loyal, more evangelical, more likely to at least give Amazon a look whenever they were buying anything that the e-tailer might stock.

Will Amazon do it? Could Amazon do it?

So… Amazon’s new Android tablet sounds interesting, but I won’t be rushing out to buy one. It doesn’t do anything that I haven’t already got covered by my iPad and my Kindle. Lots of people will buy them, and lots of developers will develop for them. That’s all good, but I really do hope that Amazon sees their new tablet as a further tool in their toolchest, rather than a replacement for their current offering.

There are plenty of people who still want a convenient and affordable way to buy and read electronic books inside and out, and there is plenty of scope for Amazon to satisfy that demand through creative pricing of the device that they already sell.

And finally…

It’s been far too long since my last post here. I could blame school holidays (which finish this week, returning the house to me and my music, at least during the day). I could blame the pressure of paid work for clients which stays locked behind closed doors (which eases this month, as three different small-but-protracted projects wrap up). Or, all-encompassingly, I should blame there always being something else to do (including the very important doing nothing at all). I’ll try to do better.

Anyway, for those who (for whatever reason) want a little more me to read, there are always my regular slots on GigaOM Pro and semanticweb.com. For GigaOM Pro, anyone can read my daily “Today in Infrastructure” thought, or subscribe to the RSS feed of short comments on Infrastructure/Cloud Computing links from around the web. GigaOM Pro subscribers also get a Weekly Update on something timely and topical. For semanticweb.com, there’s a monthly podcast and a monthly column on something Semantic Web-related.

Or you could be one of those clients who pay to have stuff written that only you get to read.

Android this week: Amazon’s tablet spied; Samsung Note debuts; Tablets galore at IFA (gigaom.com)
Amazon Kindle Tablet details revealed (techradar.com)
Amazon’s first Kindle tablet is a seven-inch Wi-Fi model (report) (mobile-ent.biz)
Amazon prepping $250 7-inch Kindle tablet with color, multitouch support, and a whole lot more (edibleapple.com)
The many ways Amazon could price a successful tablet (gigaom.com)
TechCrunch reveals the Amazon Kindle Tablet (tuaw.com)
Amazon’s Kindle Tablet: An Android fork with disruptive pricing (news.cnet.com)
Amazon’s Kindle Tablet could slaughter the Android tablet market (zdnet.com)
Amazon’s cloudy vid-tablet breaks cover: Not an iClone (go.theregister.com)

June is San Francisco month

Paul Miller — Sun, 08 May 2011 17:48:46 +0000

Image via Wikipedia

For real-world applications of Linked Data and the Semantic Web, the long-running Semantic Technology Conference is hard to beat. For getting a real handle on the Cloud Computing landscape, GigaOM‘s Structure Conference is also a leading light.

Working across both areas as I do, these events tend to figure prominently in my calendar for the year*. Last year, both took place in San Francisco during the same week. I tried to attend both, and therefore succeeded in spending most of my week in cabs, shuttling between meetings at the two venues. I saw very few sessions that I wasn’t personally involved in, and the experience wasn’t a huge success.

This year the conference organisers have taken pity on me, and moved their events to opposite ends of June. The Semantic Technology Conference is up first, back at the Hilton Union Square from 5-9 June. Structure follows, returning to the Mission Bay Conference Centre on 22 and 23 June.

I’ll be attending both, and probably doing various official things during each event. At the moment, the only thing we’ve definitely nailed down is a special live appearance by The Semantic Link crew on the evening of 5 June; we’ll be taking a look at the highlights expected for the conference, offering some tips for those new to the event and its multitude of parallel sessions, and generally bringing our usual podcast chatter to the stage.

So if you’re in town around the time of either event, do get in touch; it promises to be an interesting month.

* Disclosure: WebMediaBrands pay me to host the monthly Semantic Link podcast, and to write a monthly column on SemanticWeb.com. GigaOM pay me to curate the Infrastructure/ Cloud Computing channel on their Pro site. I attended and participated in both of these events before that was the case, and still would today without the contractual relationship.

GigaOM Structure 2011 (datacenterknowledge.com)
Semantic Technology Conference (fakeiitian.com)

Lessening the Pain of Data Roaming With Onavo

Paul Miller — Sun, 06 Mar 2011 18:58:26 +0000

Image by fatboyke (Luc) via Flickr

I am in the Belgian city of Brussels at the moment, which means that my mobile phone is ‘roaming;’ off my UK network and being charged a scary amount of money to access data. Travelling to Europe is less scary than going elsewhere in the world, as I’m ‘only’ charged about £3 per Mb here. That’s half what I’d be charged to use data in the United States, but is still an obscene amount of money for my mobile phone company to be extracting from me.

There are plenty of excuses made about termination fees and cross-border this, and intra-jurisdictional that, but at the end of the day a very small number of large telcos are getting away with what is essentially a scam. Much of the time, I’m actually connected to the local network of my own mobile phone company, rendering many of their excuses even more pathetically irrelevant than they might otherwise be.

Still, this post isn’t meant to be a complaint about mobile phone companies. I have every faith in the European Commission, and its power to bring more pressure to bear in reducing those charges. I just wish they’d get on with it.

What this post was meant to be was a complimentary post about a new startup that’s attempting to do something about this. And there’s a cloud connection, too.

Enter Onavo, a startup that uses a small app (iPhone only, for now) to reduce the amount of data you consume when roaming; thereby lowering the size of the bill your mobile phone company sends you. The company claims reductions in data consumption of “up to 80%” in optimal circumstances. I’m not seeing that big a saving, but mine is still significant.

Basically, when you turn Onavo on for a trip and start roaming, the company’s servers step in and start compressing images etc before they are delivered to a mobile carrier and — via the mobile carrier — to your phone. Compressed content means smaller content, which means less bandwidth used, which means smaller bills for me. Should you happen to find affordable wifi, Onavo gets out of the way and the uncompressed content again becomes available to you.

If you’re going to be travelling, have an iPhone, and want to give it a try, the company has given me some invite codes to share. Simply visit www.onavo.com/codata using Safari on your iPhone, and the first 100 will get into the beta programme. It’s working well for me, and might for you too.

Onavo is not paying for this post; they let me into the beta programme, I found it useful, and now I’m telling you that I did.

And the Cloud connection? Onavo’s co-founder is Guy Rosen; he of the Jack of All Clouds blog, and those invaluable State of the Cloud graphs. Speaking of which – aren’t we due another one, Guy?

Maxroam Signs Airline To Disrupt EU Cell Roaming For 78 Million Passengers (techcrunch.com)
Tighter regulation of mobile data roaming casts shadow over Barcelona from Brussels (robbratby.com)
EU wants to bring down mobile phone roaming charges (newsinfo.inquirer.net)

Trust, Big Data, Semantics, Data Marketplaces, and More Trust

Paul Miller — Sun, 27 Feb 2011 18:21:37 +0000

I’ve had a few posts published over the weekend, picking up some things I have written about before. These are;

My latest monthly column on SemanticWeb.com; Big Data Presents a Big Opportunity?
My latest weekly wrap-up on GigaOMPro; Rosslyn Analytics, Microsoft Finding Value in Data Aggregation
The teaser piece on GigaOM’s public Cloud site; In Exploiting the Data Market, Trust Is Key

I spot a theme building…

Paul Miller - The Cloud of Data » Cloud computing

CloudCamp London: the Big Data Special

Related articles

TOSCA may prove a prescient name for new cloud standards effort

Related articles

Top Level Domain for data answers the wrong question

Related articles

Nurturing the market for Data Markets

Related articles

The myth of a data free trade policy

Related articles

Microsoft, the USA PATRIOT Act, and European cloud computing

Related articles

Of Kindles and Business Models and Stuff

Related articles

June is San Francisco month

Related articles

Lessening the Pain of Data Roaming With Onavo

Related articles

Trust, Big Data, Semantics, Data Marketplaces, and More Trust