Virtual IT: Orchestration Must Change

We're now well on our journey of exploring the future state of IT in just five short years.

The first post looked at how IT's role and perspectives will likely shift over this period of time.

The second post considered the implications of when IT sees themselves exclusively as a service provider.

And in this post, we're going to start getting a bit more into the "how" rather than the "why".

We've Talked About IT Services Already

Simply put, some IT services can be thought of as "smart people who can help you out"; others can be in terms of providing infrastructure that deliver the applications the business wants, still others can be thought of in terms of overall business interests, rather than specific needs (e.g. security).

For this post, though, the starting point is on the shared infrastructure side of the discussion. 

How will shared infrastructure be orchestrated to deliver what the business demands in this new virtual consumption model?

Note that I've decided to use this term, rather than the more traditional "management" or "monitoring".  Sure, I'm always a fan of new words, but -- in this case -- it's a more accurate description of the near-term future state.

Dynamic Vs. Static

I tried to make a case at the end of my last post that -- in this world of virtual IT -- we'd see a pronounced shift to dynamic and adaptible IT, rather than static and frozen.

Getting to a good-enough result quickly will usually matter more than attempting a near-perfect answer that takes much longer.  Rarely do we do a good enough job of predicting future application requirements; why not assume that things always change, and leverage that characteristic, rather than resist it?

Hence the concept of orchestration. 

If you've ever gone to a classical music performance, the conductor mediates between the needs of the listener and the needs of the individual musicians.  Done well, it's not a static or passive role.

And it's more than just reacting to the dynamics of the music at hand.  Audience getting bored and distracted?  A good conductor will liven things up.  Individual musicians drifting off?  Get their attention back to what's going on. 

I guess that's why the good ones make the big bucks.

Getting back to IT architectures, it's easy to imagine pools of virtualized resources: compute, storage, network, etc. -- we'll save that discussion for a later post.  But, as you consider these concepts, you often end up focusing intensely on this concept of orchestration as perhaps *the* key ingredient.

And I think for many organizations, this will end up being *the* key discussion -- not in terms of technology impact, but organizational impact.  Changing technology is relatively easy when compared to changing people's mindsets.

(Virtual) Service Composition

One thing that stands out is the compositional nature of many IT services.  A business user sees an application screen, which may be composed of multiple application logic elements, orchestrated together.

These application logic elements are supported by things like databases, middleware, operating systems and the like -- also orchestrated.  And, of course, these in turn are supported by underlying technology assets: compute, network and storage.

Many IT architects speak in term of "service composition" to describe this layering effect.  Historically, this has been useful when speaking about business applications, but -- as we consider the eventual virtualization of physical assets -- it's also useful to describe infrastructure and the services it provides.

Indeed, you'll hear people talk in terms of an application service being supported by a compute service, attached to a storage service, protected by a high-availability service, and so on.

Clearly, the business value is more weighted towards the composition of things, and less by the things themselves.

But there's a fundamental problem here.  We don't usually organize IT this way.

Welcome To The "Cylinders Of Excellence"

We don't call them stovepipes anymore :-)

If you work in a larger IT shop, or work with larger IT organizations, you'll recognize the various functions.  The server team.  The network team.  The storage team.  The database team.  The application team.  The backup team.  The operations team.  And so forth and so on.

This is not a bad thing.  It makes a certain sense to align skill sets this way, doesn't it?  Back to our orchestra analogy, all the trumpets sit together, all the violins sit together, and so on.

And, as a result, server teams tend to think of optimizing server resources, and workflows and methodologies that are optimized around server requirements. 

And the storage people do the same thing. 
And the network people do the same thing. 
And the database people do the same thing.
And so forth and so on.

Now, imagine when someone wants to run a new garden-variety application.  Maybe it's a packaged application, running on a standard database, with a few exposed web services, a moderate amount of performance and availability -- nothing too special.

Think of all the teams that have be involved to drive a coordinated workflow when "good enough" would probably be good enough.  The business user is looking for a quick answer, and usually doesn't get one.

Not an entirely optimal outcome, is it?

OK, I can see the logic of doing things this specialist-centric way when it's some big, hairy, important bet-your-business application that's being considered.  But I think we're seeing a secular shift towards a larger number of smaller, potentially coordinated applications, rather than the monolithic monsters of yesteryear.

Now, it's egregious enough when we consider what's involved in standing up a new, garden-variety application.   But what happens when we throw in the dynamic element?

Hmmm -- this application seems to need more performance and protection -- right now.  Who detects this and executes this?  The need to detect and act spans the architecture, but large scale IT isn't usually organized that way, is it?

Or, better yet, this particular Really Important Application seems to be running System Idle Process as it's most important function -- why don't we reallocate its resources somewhere else until things change?

And, yes, we can imagine virtual resources that can flex up and down as needed, but who's going to be the conductor for all the musicians?

The Power Of The Model

I've written before about the singular attractiveness of a model-based approach to bridge the inherent gaps that IT will face going forward.

Models collect state information from various inputs as they're available.  They assemble and correlate this information around multidimensional relationships and causality.  And they can present their world view in a variety of modes depending on who's asking.

Need a business-process-centric view of things?  Look at the model this way.  Need a network-centric view?  Here's a dynamic slice of the same model.  Same for applications, server, storage, security, et. al.  One truth, different views.

The model becomes a singular viewpoint for all IT resources and services -- and how they relate.  And, ideally, it's not a static entity, either.  It is updated with current state/relationship information on a dynamic basis, which -- presumably -- becomes far more important as we move to dynamic and virtualized IT resources.

Now let's drive the model the other way.  A decision is made that a higher service level is needed for a business process, composed of multiple applications, and using presumably virtualized resources.  Or an promised SLA isn't being met.  Or there's a need to make room for other, more important, tasks. 

Things need to be discovered, moved around and rebalanced.

What entity can both discover the "big picture" of how these layers are related, and -- more importantly -- look out horizontally at each layer of resource?

That's why models are so intriguing and ultimately compelling -- they appear to be a fundamental construct for orchestration both upwards and downwards in the stack, and thus an essential component of future-state IT.

Back To Virtual IT

So, we likely have a third meaning for "virtual IT" among many potential meanings: an abstracted, coordinated and correlated view of all IT resources and processes that serves as the basis for orchestrating virtual resources around dynamic requirements.

Or, simply put, model-based management.

Which Brings Up Two Interesting Questions

First, which of these IT management services do you want to do yourself, and which ones might you want to hand over to someone who can do them faster/better/cheaper than you can? 

As an example, would you like to individually manage all your backup processes, or simply get a daily report from your external management service provider that made sure they all got done?

And, second, who gets to "own" this uber-console?  The server team?  The network team?  Someone else?  Something new?

We'll save those discussions for a later post :-)

Courteous comments welcome as always!

Comments

[Greg Bowden]

Hey Chuck,

What was in your cereal this morning? I have no chance to keep up with your recent rate of postings! :)

[Chuck Hollis]

Hi Greg -- hope you find the discussion interesting.

Frankly, I'm just getting warmed up -- there are about 6-8 more posts queued up.

Can't spend all my posts poking at other storage vendors :-)

-- Chuck