Hadoop made easier, courtesy of Red Hat and Hortonworks

If Hadoop has a problem, it can be expressed simply: Its power is inversely proportional to its ease-of-use. It's hard to set up, hard to work with, and hard to get real business value out of -- yet it's regularly pitched as the Next Big (Data) Thing for wringing ROI out of one's cloud.

Red Hat and Hadoop vendor Hortonworks have both been trying to assuage this problem. In Red Hat's case, it's building support for Hadoop into Red Hat Enterprise Linux as tightly as it's built in support for OpenStack. Now both companies have announced an alliance that aims to make Hadoop deployable across both private and public clouds.

Billed as "Enterprise Apache Hadoop for open hybrid cloud," the idea encompasses more than delivering Hadoop as an asset that can be expanded on-demand or moved as needed from a private cloud into a public one (or vice versa) via Red Hat's OpenShift. It also includes enhanced integration with Red Hat's JBoss family of data tools, to allow those already familiar with that toolset to leverage Hadoop as a data store alongside more conventional data sources.

Hortonworks, in turn, can ladle on its own special sauce: a currently in beta version of the Hortonworks Data Platform, its distribution of Hadoop, which will integrate with Red Hat Storage. The latter abstracts different kinds of storage, from Hadoop itself to OpenStack Object Storage, so data architects can (in Red Hat's words) "speed the development and deployment of new and existing analytic workflows."

A common question -- asked during the Q&A in today's press event -- was why Red Hat and Hortonworks chose to buddy up. The short answer: They're a lot alike. Each is noted for taking a major open source project and turning it into a useful enterprise product, without unduly compromising either's open source origins. With Red Hat, it provided enterprises with a Unix that was better (and cheaper) than Unix; with Hortonworks, it's put analytics into the hands of a market that couldn't afford it before.

It's been speculated that Red Hat would end up buying a partner outright for its Hadoop work, but that makes little sense for such a tightly focused and fiscally responsible company. Collaboration between the two looks like the better deal for both.

Hortonworks' collaborative approach also stands in contrast to Cloudera, the other big Hadoop vendor. Rather than talk about partnering with other open source entities, Cloudera's been playing up how it can challenge upscale big data outfits like IBM and how it plans to use Hadoop as the foundation for an ambitious enterprise data hub program. But not everyone who wants to use Hadoop is on board with such a program.

What could go wrong? One possible issue is how the Hadoop application envelope -- what's done with it and by whom -- will be pushed. Ex-ReadWrite COO Bernard Lunn has written about how "breakthrough killer apps for Hadoop will be created by startups and they won’t pay for a supported distribution. Hadoop's open source license is more permissive than Linux, so many companies don't need the paid version." In his view, the people who will push Hadoop the furthest are the ones who aren't going to shell out for it, but rather will build new things with it from the inside out.

Fair enough, but there's still plenty of room left for companies that want to leverage Hadoop in a way that may not be bleeding-edge to the rest of us -- and are willing to pay for it so that they can build their business on top of it. Red Hat and Hortonworks seem well on their way toward making that crowd happy.

This article, "Hadoop made easier, courtesy of Red Hat and Hortonworks," was originally published at InfoWorld.com. Get the first word on what the important tech news really means with the InfoWorld Tech Watch blog. For the latest business technology news, follow InfoWorld.com on Twitter.