« When Big Data Met Security: Is The New Era Beginning? | Main | Making A Case For Cloud: Perspectives From An IT Practitioner »

August 05, 2011


James Governor

i am always a LOT more skeptical about partners as references than customers. but frankly customers get all sorts of advantages when they are references too...

Greg Knieriemen


You know my thoughts on this... the "pay to say" game is out of control in our industry. I would also add the "pay to attack" game is out there too.

However, I blame vendors including EMC for letting this get out of hand. I know of specific "pay to say" campaigns that EMC seeded with select analysts (I even mentioned one example not long ago to you).

First... make sure EMC insists on disclosure from the analysts they engage with. Currently EMC does not insist on this. I'll be glad to name specific analysts for you who dodge this all the time. You know who they are, I'm sure.

Second... don't split hairs on the how much coaching was done with the analysts you engage with. If you are paying them a retainer or flat out fee at all and they write on a technology or specific solution to EMC's benefit, make sure they disclose.

Third... call out other vendors that abuse this like you have here. There are no specific laws yet on the games analysts play so peer review and challenge is important.

Thanks Chuck, I'm glad that you continue to have the courage to raise this issue.

Greg Knieriemen

I'll go a bit further with this... I have a challenge for you Chuck and your team at EMC. Let me be clear though, this is NOT about EMC, every vendor including NetApp, HP and IBM plays the game. I'd like to see this concern raised to action:

I challenge EMC to insist that Gartner discloses that EMC is a client on any reports Gartner issues that mention EMC or... EMC stops being a Gartner client.

There it is. Let's see if we can get the other vendors to agree to the challenge as well.

Tom Petrocelli

A couple of comments...

I think you mash two different things together that really don't go together. Talking up analysts and journalists to get coverage is not the same thing as having a mysterious organization write a skewed report for you.
Good, independent analysis and opinion is important so that good decisions can be made. As someone who has been in IT, I can tell you that you can't trust what a vendor says since they're trying to sell you something. You need an outside voice.
The central issue is the ethics of the that outside voice. Is truth important or just money. Vendors and IT folks have to judge for themselves. I personally have tried to only give an honest opinion and only affiliate with others of the same ilk. The pressure from vendors not to speak truth is enormous.
Vendors should look inward first. Do you pressure an otherwise honest analyst? Maybe even threaten them with a loss of business? Do you cross the line between advocating for your company and products and telling lies? I've seen all of these behavior as a vendor and analyst/consultant. You would be surprised at how aggressive some folks get.

Jane Rimmer

I think Chuck raises some good points, but the bottom line here is everyone seems to have a different definition of integrity IMHO. I think any customer reading a Gartner report that mentions EMC (or any other large vendor - as well as many smaller vendors) would be pretty naive if they didn't think that EMC was a Gartner client ;-)

Chuck Hollis

Guys, all I'm arguing for here is more transparency and disclosure without delving into subtle ethical and honesty discussions. There should be no problem in simply stating the facts.

And, Tom, I agree that independent voices are an important part of the landscape -- no argument there.

For example, for EMC to disclose "here's the $$$ we pay analyst XYZ every year, and what we bought from them" is not a bad thing.

Using Gartner as an example, we buy a lot of their research every year, as do many other vendors. Whether or not that unduly influences what an analyst says is a far more subjective discussion.

And having analyst XYZ disclose "here's the $$$ we get from EMC every year, and what they bought from us" would be an equally good thing.

Good discussion, thanks

-- Chuck

Greg Knieriemen

I agree - getting into "subtle ethical and honesty discussions" is dicey. All I'm suggesting is a client disclosure... the end user reading the report can conclude for themselves the value and independence of the information. BUT they absolutely need to be informed of the client relationship if it exists.

I wouldn't go so far as putting a dollar value on it. As a matter of business practice, Gartner shouldn't be expected to disclose dollar value of clients any more than EMC would disclose the street price for every VMAX sold. A simple client disclosure statement is all that is needed.

Jane: I wouldn't assume end-users understand the analyst games. Some do but many do not.

Chuck Hollis

Disclosing that a business relationship exists is probably a good middle ground.

I'll probe on the EMC angle -- that we should periodically publish a list of all analysts that we have a business relationship with. And, as an industry leader, if we do it, others are very likely to follow, which would be a good thing.

It'd be some additional work, but it shouldn't be too hard to do.

No guarantees, though!

-- Chuck

Martin G

Well, it's certainly got to the stage where I read all industry analysts with a certain amount of suspicion; I don't take Gartner's Magic Quadrant seriously at all. But I know plenty of people who do; intelligent, smart people who ought to get out into the real world a bit more often.

So I talk to my peers and even to my contacts in the vendor's themselves; they know me and can be amazingly honest about the capabilities that their products have. We have many more tools and avenues to make intelligent and informed choices than we have had in the past.


I don't think all publishers and analysts are like that. If a journal or a website contains too many sponsored analyst pieces, its already an indication that slowly and surely the publication would move into oblivion. It takes a 2-3 minute read of any article to determine if it was sponsored or not and when customers start reading too many advertorial articles, they would be put off and would start looking for meaningful stuff elsewhere.

In this game, everyone loses. The company that paid for the 'analyst piece', the analyst 'platform' and the analysts 'themselves'.

David Rayner

Pay-for-say part two...maybe folks in said marketing departments were fans of "Tin Men"...a timeless calssic:)


I agree totally that the "pay to say" phenomenon is out of control and has been for a long time. It's also disrespectful. As a result, savvy end-user customers have learned to talk to other customers, without vendor assistance, to get at the truth.

There needs to be a lot less effort at persuasion in sales and marketing and a lot more focus on clarity and just treating people the way we ourselves would like to be treated. Sustainable brand reputation depends on trust and trust is broken when vendor pay to say.

Joe Franks

Marketing is no more than a door opener for the consultative sales process to start. A responsible CIO would make the buying decision based on the merits of the proposals and verifiable claims for the product and services.

As for pimping the vendors product in return for a discount, it opens up some dangerous and possible hostile territory.

James Governor

fwiw redmonk has practiced full disclosure about our clients since we founded the company in 2002, and on our blogs since we launched them in 04. we don't publish dollar amounts for every client but our sponsorship structure gives a pretty clear idea how much a given company is paying us.

Steve Kenniston

All, sorry for jumping in late, I was too busy paying someone to review my product :)
I will say that I think we are all missing the point. The reason why vendors do this is because they don't allow any entity to purchase their equipment specifically for testing. I was the analyst who started the ESG Lab group. In the beginning I wanted to do head to head comparisons (I think between a Symmetrix and an HDS system) and EMC told us (ESG) that we COULD NOT do that. Now some of it was there was fear they may not perform so well. The other fear is that, and rightfully so, that if the head to head tests were not comparing apples to apples then it would be unfair to have a report in the market that may not be truly accurate. Yes, customer environments are all different and there is no one size fits all "test", however I think the only one who loses here is the end user.
Instead of talking about how to word the disclosure so that the vendor still gets what they want, which is a positive review of their product, I challenge each vendor to give a system or even a set of systems to a truly independent lab and define a set of test that everyone can agree on and test the products head to head. Guess what, we may learn that someones product isn't really that great for a particular environment. Then guess what, the vendor will go back and "fix" or make their product better. Who wins? The customer that is who - and isn't that who we are supposed to be helping?
Until this happens, the reality is REAL customer feedback, even from places such as TechValidte are much more authentic than some Lab report where someone comes to the vendors lab and test the vendors product in a predetermined configuration that has been set up to show the product in the best light. (Don't get me wrong, I have done it as well and given how the industry operates it has some value.)
And Chuck, you were a little over the top throwing IBM under the bus around its disclosure. The EMC FUD is thicker than molasses (and don't forget, I used to work there so I know the "spin-o-meter). If I think back just a few months, your new CMO paid almost a million dollars to a bunch of high school kids to click on a link to watch a silly Erik Estrada video to get your hit rate up to show how "cool" EMC was. Where was the disclosure there?
Anyway, keep it up Chuck - it is always good to debate this stuff and see how we can actually drive the market to help out our end users.

Chuck Hollis

Wow, Steve, what a hot mess you've thrown up here ....

Let's start at the beginning, shall we? EMC invests tens of millions of dollars of years in competitive product testing, and have done so for two decades. We consider our expertise and capabilities in this area quite substantial and very difficult for a small-scale operation to replicate in any meaningful fashion.

If I remember the original ESG proposal, it was an opportunity to loan ESG a bunch of equipment, support the effort with our own engineers, and pay ESG a substantial fee for the privilege.

Thankfully, we were wise enough to decline at the time.

Just so we're clear on the facts, anyone who buys a system is free to do whatever they like, include test it against other systems -- even publish the results if they'd like. There are no prohibitions against that activity.

However, we don't usually support free extended loaners of million-dollar arrays for that sort of exercise for anyone who thinks they have a great idea.

Call us crazy ...

Meaningful head-to-head testing is incredibly valuable. It's also incredibly difficult, not to mention expensive. We have yet to meet any organization with better capabilities and resource than our own, so -- for now -- we mostly choose to pass. However, if there's someone out there who can bring more value to the table -- we're all ears.

We go out of our way to have our customers share their experiences both with our products and those from other vendors. If you head over to our 250,000-member community site (communities.emc.com) you can see many IT professionals discussing the pros and cons of various products in an open forum.

And, other than professional decorum, people are pretty much free to say what they think.

I don't know where you came up with the million-dollar-for-clicks story. Makes for nice bar talk over a few beers, but that's about as far as it goes. If we're going to be rumor-mongering here, I can be creative and am more than willing to supply a few on your behalf.

Thankfully, you end on a positive note -- the goal being debating meaningful topics that move the industry forward.

I'm looking forward to you going more down that path, and not the one you're on.

-- Chuck


Chuck, to kill your curiosity:
Run a domain search on http://domains.whois.com/domain.php for the itgforinfo.com domain and look for the "Admin Name".

Good luck with the blog! ;-)

Hugh Wason

" erroneously implying that the new XIV and the EMC VMAX are functional equivalents (they're not), and then Comparing TCO with an uneven mix of real-world and "estimated" costs...

Wow! Quite some statement, without enlightenment Chuck... XIV, an architecture EMC Bloggers have so little regard for, sure comes up often enough.

Your Friday Rant uses this ITG report as an example of 'Pay-for-Say'and suggests misuse / inaccuracy, how about substantiation rather than Innuendo? a 'head-to-head' apples-for-apples comparison document from EMC? who supposedly have the Labs.. and access to every technology in the market, surely this enables EMC to take the opportunity to truly counter the ITG report.. lack of this begs the question... as to why nothing published... other than the odd Blog Rant ?

Chuck Hollis

Hi Hugh

First, a bit of housekeeping. If you work for a competing vendor (like IBM), or are affiliated in some way with a competing vendor, it's good practice and good ethics to disclose that affiliation prior to weighing in.

Now, on to the substance of your complaint.

Look at it from EMC's perspective. A smaller, less successful competitor (IBM) pays an "analyst" to do a hatchet job on the market-leading product, e.g. EMC's VMAX. If you research the entirety of the "analyst's" work product, it seems to be entirely pay-for-say pieces on behalf of IBM. The individual goes to great lengths to hide his real identity, business model, etc.

Surely, not in keeping with IBM's generally high ethical conduct standards?

The document starts with an erroneous comparison of the two products as functional equivalents (not true in most customers' eyes), and then tries to create a cost-comparison case using no verifiable details.

And EMC is supposed to invest resources in responding to this sort of stuff?

I would offer that the burden of proof lies with the individual making the outrageous claims. Maybe if you paid him more money, you'd get better work. Me? I'd ask for a refund.

My advice to the IBM storage team?

Go with your strengths, e.g. affinity with IBM products, supported by IBM, etc. The XIV as a standalone proposition is hardly a compelling offer, especially in light of a very competitive storage market. However, in the context of an all-IBM solution or offer, it becomes marginally more attractive.

To be clear, we at EMC welcome strong competitors with good offers that make us sweat at doing a better job. There's no need to synthesize specious marketing deliverables that insult everyone's intelligence -- not good for customers, not good for the industry, and ultimately not good for IBM.

-- Chuck

The comments to this entry are closed.

Chuck Hollis

  • Chuck Hollis
    Chief Strategist, VMware Storage and Availability Business Unit

    Chuck works for VMware, and is deeply embroiled in all things software-defined storage these days.

    Previously, he was with EMC for 18 years, most of them great.

    He enjoys speaking to customer and industry audiences about a variety of technology topics, and -- of course -- enjoys blogging.

    Chuck lives in Holliston, MA with his wife, three kids and four dogs when he's not traveling. In his spare time, Chuck is working on his second career as an aging rock musician.

    Warning: do not buy him a drink when there is a piano nearby.
Enter your Email:
Preview | Powered by FeedBlitz

General Housekeeping

  • Frequency of Updates
    I try and write something new 1-2 times per week; less if I'm travelling, more if I'm in the office. Hopefully you'll find the frequency about right!
  • Comments and Feedback
    All courteous comments welcome. TypePad occasionally puts comments into the spam folder, but I'll fish them out. Thanks!