Brian Biles' Blog

1+1 = 3 Even Before OpEx Discount

The big story in backup software is that it’s not just software anymore. Apparently my prior blog’s last line (below) was prescient. Maybe it was the memo itself.

Increasingly, the new field of battle is in deeper integration between backup software and dedupe storage. Avamar started this wave a long time ago, and it remains the most advanced implementation. More traditional backup software is also moving in this direction:

• Symantec pioneered the idea using OST and Data Domain Boost to integrate NetBackup, and now Backup Exec 2010 with Data Domain and other OST licensees.
• Starting this summer, NetApp and SyncSort are offering channel-based integration packages for SyncSort to manage snapshots on FAS systems. Not quite Boost, but reacting to the same problem.
• Also starting this summer, Symantec are offering their own storage systems in some geographies for PureDisk. (I love OST, but how weird is it to be on the NBU 5000 team in the “No Hardware Agenda” company? It would be like working in DB2 support at Oracle.)

As noted below, EMC has moved aggressively to integrate NetWorker and Data Domain based on the Boost technology at the Storage Node (backup server) level. This is now available in the latest release of NetWorker. Avamar and NetWorker have already been integrated for some time from the NetWorker client.

Integration brings speed, as discussed below, but it also brings lower OpEx. Following up on the launch last May of NetWorker using Boost to integrate with Data Domain, we looked in detail at some of the results in our beta sites. To these customers Data Domain was already fast, but being able schedule replication, manage multiple retention policies, clone to tape as well as monitor and report on Data Domain systems all within NetWorker also brings significant efficiency improvement. One customer said they expect to see a 20-30% reduction in management time.

See prior memo.

Storage Vendors Stages of Grief About Deduplication

Deduplication is a challenge to most large storage vendors. IT means they get to sell fewer disk drives, and their business plans involve more drives. I expect most vendors will go through Elizabeth Kubler-Ross' five stages of grief in confronting this dilemma. You can already see it in practice in some of the biggest vendors.

Denial:

• Internally: Deduplication is too slow and it puts data at risk. It ain't real storage, it doesn't work. Their implementation is risky!
• Tell customers: How about a free VTL with your next SAN/NAS upgrade? Let's get lunch.

Anger:

• Internally: Who do they think they are? Dedupe is a fad! Real storage is tons of platters, spindles, lots of sheet metal. This whole industry is going to hell! We need to muddy the water. Buy or OEM something fast and hopefully cheap. Start blogging!
• Tell customers: How 'bout a deal on your next VTL expansion? Running out of space? Right now we can discount VTL really low. I wouldn't bring this up, but I see you're talking to Data Domain...Did I tell you we have box seats at the game next week?

Bargaining:

• Internally: We need to do something. Our sales people are screaming bloody murder! Can we resell something cheap? One size does not fit all.Toss something out we have lying around in the labs. We bundled it in enough deals to tell IDC we have market share, so stop bugging us.
• Tell customers: Please spend a month to review our VTL. Look, it has a little deduper in it. No? Take some more time, look at out esoteric backup software - it has that dedupe thing you have been asking about! No? We can get you a deal on it with that SAN upgrade we have been talking about. We totally understand your needs. Use some professional services hours. Oh, you need our challenging dedupe NAS. No?

Depression:

• Internally: Crap, this stuff isn't going away. What, they went public? We let the market decide, and it turned out we got weak products, by (choose one) trying to preserve and bolt on to our existing designs - or - we OEM'd weak products for time-to-market from vendors who were desperate. (Sigh)
• Tell customers: What's the difference anyway? It's just a feature! What? It's not? You need to test it? Look at this roadmap. This PDF says we check almost as many options as they do. I'm sure we have a reference somewhere...

Acceptance:

• Internally: Wonder if they are accepting resumes?
• Tell customers: We are working on something else, we need more time. Wait for us! I'm serious, our backup software may be less fussy now. We can talk over lunch.

Nothing Has Changed

Last week, Data Domain announced the highest performance inline deduplication system in the industry with the DD690. EMC and Quantum followed hastily with their own announcements.

Nothing has changed

EMC is repacking the Quantum 7500 in the EMC DL3D line. But the inline dedupe rate is still about 150 MB/sec., according to this discussion, after which the only rates we can know for sure are to and from their non-dedupe cache. So the effective capacity, the amount you can dedupe in a backup day, is something less than 13 TBs (dedupe rate x 24 hours). If you don't finish deduping in a day, it will compromise tomorrow's throughput and backup window. So the only way you could use 150 TB of this disk for dedupe is if your backup window is more than two weeks long.

Why offer 148-180 TB in this architecture? Because it is not a dedupe system, it is a traditional VTL (even if it has a NAS interface) -- a disk-based, short-term-retention I/O buffer for data on the way to tape. The extra storage baggage? Call that dupestorage. It is not meaningful for sizing dedupe or replication. Data Domain's dedupe speed and resulting effective capacity is greated by more than a factor of two.

Seems like everything's still the same:

• Data Domain is still the deduplication performance leader;
• EMC and Quantum are still selling traditional VTLs based on dedupe storage;
• EMC is still selling massively over-provisioned disk systems;
• Quantum is really trying to sell tape libraries above all.

Oh, I guess one thing has changed. Inline deduplication used to be considered too slow compared to a post-process. I guess that has changed. The non-dedupe write speed of the EMC DL3D 3000, at 1.44 TB/hour, s about the same as the inline dedupe speed of the DD690. If post-process isn't even faster than inline, why bother?