« Dear Clouderati: Enterprise IT Is Different | Main | Never Waste A Good Crisis »

June 19, 2014


John Nicholson

I saw a case not too long ago where a Storage Admin "went postal" and killed the storage array's and their snapshots and replication. The "tape" that was going offsite he was also responsible for and backups had been failing. To top it off he killed the HVAC in the datacenter.

Its not just the cloud that makes us vulnerable, its any kind of single access, single person monitoring of storage and BR/DC that makes you vulnerable. I suspect this is why a lot of companies use a cloud based DR/BC solution when they are too small to have proper separation of duties and oversight.

Abdul Jaludi

I wonder how many companies out there simulate an actual disaster as part of their recovery testing. I've seen too many companies copy missing disaster recovery data from production in order to say they successfully tested their recovery process.

No one expects an actual disaster to occur, or if it does, think they'll be able to access production data. Unfortunately, it is a lesson learned too late, then repeated over and over again when times are good and budgets are tight.

The other thing many companies don't plan for is the loss of key staff. Backup data is worthless without the people who know how to restore it.

Abdul Jaludi
Author of Command Center Handbook

Chad Sakac

Disclosure - EMCer here.

Chuck - a valuable lesson for all, to be sure.

The point on Snapshots (that are not separated and secured - which can be done in a myriad of ways) is one to heed.

I think though, that if people take away from this (I doubt it was your thrust) that "AWS = bad". The fact that they used simple pwd security should be the first big no-no. I think to not use MFA (the AWS Multi-factor authentication service) is a bad idea unless you're just messing around.

I feel very sad for these poor folks (and their customers).

The comments to this entry are closed.

Chuck Hollis

  • Chuck Hollis
    Chief Strategist, VMware Storage and Availability Business Unit

    Chuck works for VMware, and is deeply embroiled in all things software-defined storage these days.

    Previously, he was with EMC for 18 years, most of them great.

    He enjoys speaking to customer and industry audiences about a variety of technology topics, and -- of course -- enjoys blogging.

    Chuck lives in Holliston, MA with his wife, three kids and four dogs when he's not traveling. In his spare time, Chuck is working on his second career as an aging rock musician.

    Warning: do not buy him a drink when there is a piano nearby.
Enter your Email:
Preview | Powered by FeedBlitz

General Housekeeping

  • Frequency of Updates
    I try and write something new 1-2 times per week; less if I'm travelling, more if I'm in the office. Hopefully you'll find the frequency about right!
  • Comments and Feedback
    All courteous comments welcome. TypePad occasionally puts comments into the spam folder, but I'll fish them out. Thanks!