All of us in the IT industry have a peculiar behavior.
A buzzword spontaneously appears, we all echo it fervently, but we're not really 100% sure what it all means.
At some point, we remember that customers buy solutions to business problems, and don't spend a lot of money on buzzwords, so we get down to brass tacks and figure out what this stuff might really mean, and what business problem it might solve.
Grid is a reasonable case in point. Just getting to a common definition is difficult. And trying to figure out "OK, exactly what problem does this solve?" gets even more interesting.
So let me try and wade in.
Different Views of Grid
Now, please remember, there are people with far more expertise than I have. All I claim to be is an interested bystander, so let me tell you what I see, and see if you agree. And, my apologies to the good people at Oracle who may have a slightly different view of grid.
The Utility Computing View
One popular view is a dynamic resource-on-demand view. A big glop of CPUs and storage; applications take what they need, give it back when they're done, and so forth. Hopeful benefits: more efficient utilization of resources, and no need to size applications rigorously.
This view hasn't really taken deep root, even though this has been discussed for a while. First, there's a natural reticence for business owners to share infrastructure, although this can be overcome. Second, there are enormous challenges in allocating resources dynamically, managing service levels and so on. It looks like you can only get so far before applications have to be modified, and that's something not many people want to tackle.
Also, remember the key payoff here is saving money. Nice, but not enough to really move a lot of people forward.
The Consumption Model View
Sun took this a bit further and proposed a utility pricing model to go with the utility computing model. I think it was something like $1 per CPU hour and $1 Gb/month. I'm sure that's not what people actually paid (do a quick calculation, and you'll see it's pretty expensive), but the idea was to rent capacity to people who needed it.
Clever idea, but I haven't seen it set the world on fire just yet. I think the reason is that heavy users who need this kind of scalable crunch are also in a position to buy the car, rather than just call a taxi.
The Partitionable Application View
Another view of grids is that it's really driven by the application, not the architecture. I fully subscribe to this view. If it's a tool, then it's there to solve a problem -- understand the problem and you can better appreciate what makes a good tool.
One class of grid-friendly application seems to be what I call partitionable application. The work can be easily divided up into equal size chunks, and little interaction is required between the application threads.
Many years ago I played with POVray -- it's ray-tracing freeware that produces some pretty spectacular graphics. It's a partionable application -- each CPU can do its part of the picture, and there's no interaction between the threads. All you need is a common information view (file system) and something to coordinate the overall activity and glue all the pieces together when you're done.
There are others -- some forms of data mining are partionable applications, weather modelling probably fits in this category, and so on. Interesting computing problems, but not exactly an answer to broad-based IT challenges (unless, of course, you're Pixar). Lots of this stuff out there, but it's relatively mature at this point.
The Coordinated Chain View
OK, we're going to need to come up with a better tag line, but there are a certain class of grid applications that aren't very partionable, but still very important to consider. This is where I think the action will be.
Imagine you're a credit card company. It's time for your monthly direct-mail shot. Everyone of those mailers costs about $2.50 to send out, and there's 20 million of them, so getting it right is rather important.
From a pure IT perspective, this is a mammoth undertaking. You'll need dozens and dozens of information sources -- huge files. You'll need to scrub the individual data sources. You'll need to correlate them over and over again in different ways. You'll need to do predictive modelling against likely hit rates against different classes of names. And you'll need to manage history so that you'll know what worked and didn't, so you can do better next time. Get it right, and you're a hero. Get it wrong, and a lot of mail ends up in the trash can.
I think of it as dozens of large-scale IT applications, each potentially ravenous consumers of CPU and memory, running sometimes in parallel, sometimes sequentially. You'll need to dynamically assign CPU and memory. Lots of high-performance storage. You'll need some pretty sophisticated tools to manage workflow and service levels. You'll need a large, common view of shared information across the complex.
Now, in my mind, this is something closer to what more businesses will want. The profile is a major data analysis task, many different styles of data inputs, lots of massaging and manipulating, and -- of course -- high economic consequences for getting it right (or not!).
Just to be clear, we're not talking classic data warehousing or business intelligence here -- we're talking large-scale data grinding that isn't supported by these computing models.
Now that I'm looking around for these kinds of beasts, I'm seeing more and more. As an example: anyone who has massive customer (or prospect) lists. Large-scale e-tailers and web properties. Certain sensitive government activities. Certain types of well-field analysis in the energy business. Certain aspects of financial modelling. Every few days, there seems to be another one.
I think what differentiates these kinds of applications are (1) they need enormous amounts of CPU, memory and storage, (2) they're "lumpy", e.g. not easily partitionable and inherently well-behaved, (3) they're coordinated, in that things need to be orchestrated with a higher-level task in mind, and (4) there's serious money on the table in terms of doing it right.
Where Is Grid Going?
In a nutshell, I believe grid will be defined by the type of application characteristics it supports. Before long, grid will only be usefully discussed in terms of a specific application context.
I also believe that the focus will move from the basic computer architecture (how did you lash it all together?) to the management and orchestration of the resource.
How are resources dynamically allocated? How are service levels managed? How is work product protected and recoverable if there's a problem?
I also think that -- while grid-enabled application building environments exist -- there'll be more interest in taking existing application logic, and stitching it together with other pieces to achieve the overall goal. Simply put, I don't think too many people are going to want to rewrite their tools to exploit grid.
And finally, I think the real interest will be in massive data analysis applications that really matter -- and I've outlined a few here.
So what is EMC doing?
Well, more than you might think.
I'll save that for another post ... :-)
Comments