Cloud computing is a hot topic today, and so all sorts of companies, technologies, and industries are trying to align themselves within the framework of cloud computing. Two common terms that people often associate with cloud computing are grid computing and utility computing. But, it’s often unclear how cloud, utility, and grid computing relate to each other—if at all. It’s also often unclear what people mean when they use these terms. For example, the US Government has a two-page document trying to define just cloud computing.

One way to sort through all this confusion is to go back to origins. Because these terms are used so commonly now, it’s easy to forget that cloud computing, grid computing and utility computing are metaphors. They are figures of speech designed to help us think about different concepts. So, rather than go into great detail in defining these concepts (we’ll leave that to the US Government), let’s take a look at what these metaphors indicate and see how that helps us think about how cloud, utility and grid fit together:

• Cloud computing: Cloud computing is about treating your computational workload like a cloud. It’s out there in the sky, somewhere—distant from you. It’s sort of fluffy and fuzzy, and you don’t care too much about its details. It’s just a big white blob, and the point is that you don’t need to know any more about it. Also, because you don’t need to know any more about it, this means that you can run any type of workload or heterogeneous job in the cloud. You shouldn’t be locked into a certain set of capabilities by moving to the cloud.
• Grid computing: Grid computing visualizes a large two-by-two lattice structure—a grid! It combines a large number of individual computational nodes and assembles them into a larger, cohesive unit which can solve a computational problem. Grids are typically uniform in structure, and grid computing implies distributing a computational job across a wide number of individual, uniform computers assembled together. Grid computing visualizes the interlinks between a large number of computers assembled together for a single purpose. It is good at solving problems that distribute out amongst its individual nodes. (Note that node is itself a metaphor. A node is a “centering point of component parts.”)
• Utility computing: Utility computing is about treating computing power like a water utility or electric utility. Computing power is a commodity you can turn on and use as you need it. Then, at the end of the month, you pay a bill for how much you used. If you require a lot of computational power, you just turn on more and pay more. If you don’t use any of your utility, you might pay just a maintenance fee. Additionally, you can use your utility for whatever purpose you want. When you buy electricity from your electric utility, it doesn’t matter whether it’s for powering a CD player, a stove or a shaver—it just works. Utility computing should be the same.

Now that we’ve loosely described these terms, let’s see how they might fit together and how we can think about technology solutions in their context. First of all, it should be clear that cloud computing, grid computing and utility computing are not the same thing—even though lots of vendors would like to fit their products into the cloud computing category. But, it should also be clear that these things can work together. If you’ll excuse the mixed metaphors: A Grid of Clouds can provide a powerful Utility.

Let’s make this concrete by using Red Hat Enterprise MRG as an example. Red Hat Enterprise MRG integrates Messaging, Realtime and Grid technologies to provide a powerful distributed computing platform. MRG’s Grid scheduler, which is based on Condor, is designed to provide the ability to schedule any computational job or application to virtually any computational resource available—including public clouds like Amazon EC2, private/hybrid clouds built on virtualization technology or bare metal machines like dedicated servers or idle desktop workstations.

Let’s say that you have an application, say a Web server, that you want to run in the cloud. In other words, you want to be able to run the Web server but not think about where it’s actually running—it’s out there somewhere doing what you need it to do, but you don’t really care about the details or managing the underlying infrastructure yourself. You also want to run this Web server as if it were on a utility—if you need more Web servers, you just plug them in. And, you don’t want to have to plug your Web server into a Web-server-only jack—all your applications should work the same way in drawing computing power, and they should all work with the amount of power they require to function well.

Red Hat Enterprise MRG aims to allow you to accomplish all this by using MRG’s grid scheduler to schedule your Web server and other applications to run. MRG will then go out and acquire appropriate resources for you to run these jobs. It could be in your own private cloud built with Red Hat’s virtualization technology since MRG integrates with virtualization. Or, it could be on bare metal, or it could be at Amazon EC2 if you have no available capacity in your local data center. If you need additional Web servers, you can just schedule them with MRG’s grid scheduler and it will run them for you on an appropriate resource. If you need to consolidate or shut down Web servers because you no longer need as much capacity, MRG can manage that for you. And, if you have a job that needs to use tremendous computing power at once (many computers behind the scenes), MRG is designed to do that for you too because grid schedulers are good at that.

Going back to the concepts of clouds and grids, MRG demonstrates that a cloud provides access to a set of resources for an application. In this example, that set of resources is Web-serving capacity—whether at Amazon EC2 or in a local data center or elsewhere. A grid in this case, then, is a specific set of these web-serving resources. And, MRG’s grid scheduler schedules and manages this workload across specific cloud resources.

Because Red Hat Enterprise MRG includes the ability to talk to public clouds and integrate with virtualization (a foundational technology for building clouds), it enables users to treat their clouds as true utilities—sources of computational power that are available as needed, no matter the source. So, how do cloud, grid, and utility computing relate? To mix metaphors even more, you can build a utility grid across and through a variety of clouds.

Unless you’ve been stuck on a time-traveling island for the past few years, you can’t fail to have noticed that the hype behind such “next-generation” approaches as SOA, Web Services and REST has been simplified under the banner of cloud computing. The danger is that those who have traveled the well-worn Hype Cycle will simply discard this latest set of buzzwords (e.g., Platform As a Service and Infrastructure as a Service) as just another example of analyst-driven or vendor-driven marketing. However, the reality is significantly different. As with the other “hype-driven” terms mentioned previously, cloud has been incubating within the industry and academia for many years. Its roots can be traced back beyond grid computing (data grids as well as process grids) through ubiquitous computing and beyond. For example, in the early 1990s the European Union was funding research projects into utilizing a network of PlayStation One’s as a cheap pool of processors that could host services (typically ones needed graphical processing capabilities) on demand.

So just what is cloud and why is Red Hat a key player? To answer the first question it sometimes seems easier to try to catch a cloud! Because of its background, there tend to be several different definitions of cloud computing. But in general we shall assume that cloud represents a way to provision an array of Web-based services aimed at allowing users to access a range of capabilities on a pay-as-you-go basis, where the hardware and administrative investment is met by the cloud provider. This is in stark contrast to the traditional methods, where the end user had to provision and manage the hardware and software required to execute these services. With cloud comes the ability to give to end users across the world the illusion of low-entry-barrier, virtually infinite compute power at a fraction of the cost (in fact they only pay when they use this power and no longer need to be concerned about hardware investments languishing idle for most of the time.)

From an IT perspective, this is incredibly important. Very few companies can afford to buy all of the hardware they need, let alone administer it 24×7, especially as they grow. Peaks and troughs in usage make it difficult to know precisely what hardware is required to obtain an optimal level: buy too much and it remains unused a lot of the time; buy not enough and you cannot cope with additional demand. Cloud offers a solution to this that allows IT to invest in their own hardware and “plug in” to the cloud for additional on-demand requirements. In fact when people talk about the cloud today they are typically referring to public clouds or external clouds, i.e., those that exist outside of the corporate firewall. However, private clouds or internal clouds (within the firewall) are more likely to be the biggest users of cloud services for very important reasons: security and trustworthiness (where your data resides is often more important than how you access it). That’s not to suggest public clouds are not useful: far from it as has already been mentioned. But cloud users who are only interested in public clouds are likely to be the minority as adoption grows. The integration of public and private clouds is inevitable.

As with real (weather) clouds, clouds don’t just pop into existence like some exotic particle in physics. They need the right set of circumstances to come together in order for them to exist. At a minimum they need the right low-level infrastructure (VM and operating system) to support clustering of machines that have to be able to cope with arbitrary service requirements (this is often called Infrastructure-As-A-Service). Then of course you’ll need a way of developing applications for the cloud (Platform-As-A-Service). From a Red Hat/JBoss perspective we are working with many others in driving the understanding, adoption, interoperability and standardization of cloud computing at all of these levels. When you deploy on a Linux-based cloud, chances are that you’re using Red Hat Enterprise Linux. When you deploy your POJO’s into a cloud, you may well be using JBoss Application Server. As with SOA, open source is rapidly becoming key to opening up the cloud to everyone and not simply keeping it as the domain of the select few. Furthermore open source infrastructure technologies for cloud computing offer transparency and view of code and APIs in an effort to ensure strong interoperability from the start – no hidden APIs, no traps. Red Hat is uniquely positioned to help drive this adoption, with components that are needed at all levels of the cloud. For example, the work we’re doing around cloud-based data caching with Infinispan, or the high-performance messaging with Red Hat Enterprise MRG (of which JBoss Messaging is now a key element). In the future we expect to make more cloud announcements as we expand our vision and continue to develop these components into solutions in collaboration with our partners. Open source is a natural fit for this new growing market area.

Additionally, we expect that other open source projects and suppliers will continue to develop cloud technologies to break down the industry’s current barriers. We are inviting participants across the industry to join us in presenting new ideas and emerging open source cloud technologies at the Open Source Cloud Computing Forum that Red Hat will host on July 22, 2009. Submit ideas to present here.

Recently we’ve seen some surprising comments about Red Hat’s stand on software patents and, in particular, about one of its patent applications related to the AMQP specification. It looks like clarification is called for. Our views and our position, as expressed in our work for patent reform, our Patent Promise, and our work with the AMQP project, have not changed.

Red Hat has worked hard to address the problems of our patent system. We believe there are serious problems with the existing system, particularly as it affects free and open source software. In just the last few months, we have supported new patent reform legislation, submitted a brief in the Bilski case arguing against patenting of software, and created an innovative patent settlement in the FireStar case that gave broad protection to the open source community. We are also proud of our work in helping establish the Open Invention Network, supporting the Peer-to-Patent program, and developing our Patent Promise.

We have also worked to build a patent portfolio. As we have explained many times, the purpose of this portfolio is defensive. This means that it is designed to discourage patent lawsuits by giving us the ability to retaliate against potential patent aggressors by asserting counterclaims as a defense. We believe it is important to have such a portfolio, because of the threats of companies that are hostile to FOSS and that have amassed large stockpiles of overbroad patents.

We have made a public commitment to this purely defensive approach with our Patent Promise, which is at http://www.redhat.com/legal/patent_policy.html. This promise is binding. It demonstrates our determination to use our patent portfolio only in the defense of free software.

Although there have been some recent questions about one of our patent applications relating to the AMQP specification, they appear to originate in an attempt to spread FUD. There’s no reasonable, objective basis for controversy. In fact, the AMQP Agreement, which we helped to draft, expressly requires that members of the working group, including Red Hat, provide an irrevocable patent license to those that use or implement AMQP. In other words, even if we were to abandon our deeply held principles (which we will not), the AMQP Agreement prevents us from suing anyone for infringing a patent based on an implementation of AMQP specification. Moreover, our Patent Promise applies to every patent we obtain. Red Hat’s patent portfolio will never be used against AMQP, and Red Hat will support any modifications to the specification needed in future to verify that fact.

Red Hat has a a long track record of supporting FOSS generally, and it has worked hard to address the shortcomings of the patent system. We intend to continue that support and those efforts.

Today we are celebrating a momentous occasion. Ten years ago today, Red Hat and IBM began our global collaborative partnership to expand the use of enterprise solutions on Linux. It was a small but important start to announce that IBM would run Red Hat Linux on its industry-standard systems. Back in 1999, Red Hat was on the eve of its IPO, and IBM was testing the waters of Linux. Only 10 million users ran the Linux operating system at the time, according to IDC Research quoted in our original partnership announcement.

The global partnership has broadly expanded over the years, and today delivers combined solutions driven by rich joint technology innovations. Together, Red Hat and IBM are deeply penetrating the mission-critical infrastructures of many of the world’s Fortune 500 companies, delivering value to our joint customers through the combination of open source solutions, comprehensive services, solid platforms and technology leadership. Red Hat stands as a Strategic Alliance partner for IBM — the top ranking partnership category — and IBM is a Premier OEM partner for Red Hat.

Our alliance helped spur broad-based industry Linux adoption, driving the one of the fastest growth rates for mainstream operating systems in the past decade. As of 3Q08, IDC’s Server Tracker indicated that Linux accounts for 14% of the overall server market. (rolling 4Q average). Red Hat is the top commercial contributor to the Linux kernel, and IBM is one of the world’s top Linux evangelists—the third largest contributor to the Linux kernel — and runs Red Hat Enterprise Linux across all of our servers and 500 middleware programs.
» Read more

Red Hat Enterprise MRG was launched at the Red Hat Summit in June 2008, providing customers with a next-generation IT infrastructure that offers increased performance, reliability, interoperability and faster computing through the combination of Messaging, Realtime and Grid functionality.

Today, version 1.1 of Enterprise MRG was released. With 1.1 comes enhanced features for each of the components of Enterprise MRG – Messaging, Realtime and Grid. MRG’s Grid technology was available only as a Technology Preview in MRG 1.0, but today is fully supported and production-ready for those with a Red Hat Technical Account Manager (TAM).

New highlights and features in Enterprise MRG include:

• Messaging: native infiniband and RDMA driver for dramatically better latency, clustering, enhanced security, queue semantics like Last Value Queue and Ring Queue, Native .NET client and improved management tools.
• Realtime: improved performance, especially on boxes with higher CPU-counts, and improved performance tools. For example, Tuna now has the ability to write tunings to an init script once you’ve found an optimal tuning for your system.
• Grid: enables enterprises to achieve higher peak capacity and IT utilization through the creation of high-performance grids by leveraging their existing infrastructure. It provides the most advanced and scalable platform for both High-Throughput Computing (HTC) and High-Performance Computing (HPC), and also offers the ability to schedule to all available computing resources including:
  • local grids,
  • remote grids,
  • virtual machines,
  • idle desktop workstations and
  • dynamically provisioned cloud infrastructure.

» Read more

The events team is gearing up for spring across the globe, and we’re excited about discussing the power of open source software at several events this month. From webinars on reducing IT costs during tough economic times, to understanding the benefits of cloud computing, or hosting our first ever virtual tradeshow, there’s always plenty to see and do at Red Hat events. Below is a list of where we’ll be and more information on attending. We hope to see you there!

Worldwide
Not able to leave the office? No worries. We’ll come to you. Red Hat is hosting a webinar on February 26 entitled “Finding new way to carve out costs safely. How trusted advisors from Red Hat Consulting make it possible.” Click here to learn more and register. » Read more

QPid is the upstream open source project led by Apache that Red Hat participates heavily in to help develop Red Hat Enterprise MRG and to provide AMQP in Fedora. The initial QPid proposal was submitted by a Red Hat engineer to Apache, and the QPid community has grown significantly since then to include a large set of diverse participants.

Building on the recent announcement that it has joined the AMQP working group, Microsoft has now announced that it will be joining and contributing to the open source QPid project at Apache to build its AMQP implementation. This is also great news for the open source world and a bold new step for Microsoft.
» Read more

Today, Microsoft announced that it has joined the AMQP working group. As a founding member of the AMQP (Advanced Message Queuing Protocol) working group, we at Red Hat are excited about this development.

Just as Red Hat has been adding native AMQP support into the Linux platform and ecosystem at Fedora and through Red Hat Enterprise MRG, Microsoft is bringing AMQP support to Windows and its ecosystem. Between Linux and Windows, AMQP will become a standard messaging facility on the vast majority of operating systems and server platforms. It will offer a new level of interoperability between Linux and Windows using open standards and open source software. And, it is designed to lead to breakthroughs in everything from core infrastructure software to management tools to next-generation applications and architectures. At Red Hat, we are already building upon our AMQP messaging implementation for everything from virtualization management to security management to monitoring.
» Read more

Today we announced the availability of version one of Red Hat Enterprise MRG, delivering on the announcement of the MRG beta made in December 2007. Our customers and partners alike have contributed immensely to the innovation that has resulted in MRG – we’ll focus on the contributions from our MRG beta customers to date in this blog, but be sure to see the highlights of our partner contributions as well.

Beta customers become involved in projects with Red Hat as early innovators and adopters of our solutions to help us define future releases and bring innovative solutions to market quickly. We’re sounding boards for each other to see what tweaks and patterns work best in different project architectures, what challenges custom workloads will face so that we can provide improved performance and a compelling solution to customers at GA.
» Read more

Today we announced the availability of version one of Red Hat Enterprise MRG, delivering on the announcement of the MRG beta made in December 2007. Our partners have made large contributions that, coupled with the innovation delivered by our beta customers, have helped us deliver on MRG today. AMD, Cisco, IBM, SUN, Intel, Amazon, University of Wisconsin, Madison and HP have all made significant contributions today.

IBM and Sun have worked closely with Red Hat on Realtime. Both are certifying their realtime Java solutions on MRG – IBM is doing so exclusively. Additionally, Red Hat and IBM have been working together over the past several years on the development of a realtime platform for the DDG 1000 Zumwalt Class Destroyer program. IBM and Raytheon are winners in this year’s Red Hat Innovation Awards for this development work.
» Read more