Daniel's Blog

Driving Down the Costs of Deduplication: "Continuing the Continuum"

Since shipping its very first deduplication storage system in 2004, each successive generation of Data Domain product has deduped faster inline and offered extra capacity (both raw and logical capacity). This "continuum" of performance and capacity increases has been marked by some true milestones - such as the recently introduced DD880, which eclipsed the single controller ingest and store to disk speeds of all post-process deduplicating VTLs (and even of many VTLs that don't dedupe at all). And yes, along the way, the features and capabilities of Data Domain systems have evolved with new capabilities (cascaded replication, OST, ability to address archival workloads, retention lock for IT governance, etc.) Putting those enhancements aside for a second and focusing purely on Data Domain's "continuum" of performance and capacity increases, what are we really delivering in terms of unique customer value? In short, it's this: driving down costs of deduplication - while keeping customers ahead of the curve in terms of their data growth challenges.

So why is this interesting? I spent all of last week at SNWin Phoenix, AZ, running the SNIA Data Deduplication Hands-On Lab. Not surprisingly, data deduplication was once again a very popular topic throughout the conference. In the concurrent sessions, many of the vendors were busy arguing about technicalities, and this can definitely make for interesting debates, as well as the occasional farce. While some lab participants were getting their first exposure to the technology, many of them were now well versed in the particulars of the technology and their interest and questions were surprisingly singular in nature. They wanted to know how deduplication can save them money in the face of rapid data growth averaging 50% a year or more.

Well, there are la lot of ways to achieve cost savings with inline deduplication technology, ranging from automation of labor intensive DR processes to the whole tape replacement story. But, curiously, I instead found myself telling these users the same story I've been telling customers since I first joined Data Domain in 2006 - namely the "continuum" of performance and capacity increases demonstrated by each successive generation of our systems. How specifically does this drive down costs? Ultimately, it's all due to our SISL scaling architecture, which leverages the upgraded multi-core CPUs and memory in these new appliances - but, for purposes of this blog post, I'd prefer to address the question at the system and real-world implementation level.

Consider the new mid-range and entry-level deduplication storage systems Data Domain announcedearlier this week (the DD610, DD630 and the DD140 entry-level system). Each of these new systems replaces a previous generation that was similarly targeted for either mid-range enterprise applications/regional data centers or remote office locations. In the case of the new mid-range systems you get 60% more capacity and 100% more throughput performance. For the new entry level remote office system, you get 100% more capacity and 50% better performance. So, cost per Gb goes down (yet again) and price/to performance improves. Users can protect 60%-100% larger datasets and they can write their data 2x faster, enabling them to meet their nightly backup windows.

Based on what these users at the SNW SNIA labs were asking me, I remain convinced that Data Domain's proven track record of ongoing significant performance and capacity increases (while relative costs decrease), may actually be the most important element of the value proposition inline data deduplication offers.

So, I guess we should "continue the continuum?"

Measuring Up in the Enterprise

In my last post, I spoke directly to the claim by competitors that Data Domain is not an enterprise solution. I indicated that the number one capability our largest customers require is immediate, reliable and efficient replication of deduplicated data, as it has tremendous cost-saving benefits for organizations that are still using tape for offsite DR. In fact, it was the focus of a presentation I gave last week to a large customer looking to expand their Data Domain implementation and standardize on our solution across all branches. During our meeting, we discussed the metric against which dedupe replication as a DR solution is measured, Time-to-DR. This is the period of time measured from the start of the first job in the backup window to the completion of replication for all jobs. The high-level steps that make up the process measured by the time-to-DR metric are:

1. Full backup
2. Replication
3. Data consistent and readable from replica

To understand how Data Domain Replicator achieves the fastest end-to-end data movement, lets work through these steps using the example of a subsequent full backup of a 20TB dataset, that is then replicated using an OC-3 (155mbps) WAN link to a DR site.

1) Data Domain inline deduplication is capable of backup throughput of 2.7TB/hr, so 20TB can be completely backed up within 7.4 hours. At a 50x reduction (combination of deduplication and compression), this will result in ~400GB of data that needs to be physically stored and replicated.

2) Because data is deduplicated inline, Data Domain Replicator can begin moving the deduplicated data immediately. This enables the data movement process to run concurrently with the backup window, as well as ensures that data is recoverable at file/image/job level as soon as it has been replicated. In this case, replication over the OC-3 can keep up with the backup process and will complete within minutes of completion of the backup, and within an eight hour window from the start of the first backup.

3) By design, data is continuously consistent on the Data Domain replica - no additional steps are required to make the data readable. This means that as soon as replication completes the data movement, the 20TB of data is fully protected and available for both local restores in the primary site and recovery in the DR site in the event of a disaster.

For any enterprise customer looking at using deduplication to enable offsite disaster recovery, this best practice test of protecting a large dataset (20-30TB), replicating, and recovering from the replica should be mandatory.

WYSIWYG

This past week at SNW Spring in Orlando, Florida, Data Domain participated in the inaugural Deduplication Hands-on Lab. Our lab started with a series of excercises that had the users interacting with data in a CIFS share on a pair of Data Domain DD530 systems, with the following three learning objectives:

1. Analyze the effects of inline deduplication on files stored to a CIFS share on a Data Domain deduplication storage system.
2. Create deduplicated snapshots to provide point-in-time recovery of previous versions of files.

3. Recover data from a replicated copy on secondary Data Domain system.

Students really liked these exercises, which had them using the Data Domain systems as NAS storage - many of them followed up at our in-booth demo with F5 ARX file virtualization to get further insight into how they can deploy this technology in their enterprise. During the second half of the lab, students worked within Net Backup 6.5 to perform backup, optimized duplication and replica recovery operations using Data Domain OpenStorage software. Interestingly, while feedback to the entire lab was very positive, it is the step shown below - where students use a CLI monitoring tool to observe the real-time effects of inline deduplication - that continues to have the greatest impact on illustrating what inline deduplication really means.

With Data Domain, what you see is what you get: deduplication occurs inline, and only unique data is written to the physical disk - notice the significant difference between inbound data on the network interface (a) and data written to disk (b), and that disk activity never exceeds 2%. This exercise illustrates our CPU-centric inline deduplication at work.

Time and again, students, and even representatives of other vendors in the lab, would watch this and have their own version of an ahh-ha moment - they were seeing for themselves that with Data Domain deduplication storage, writing data and deduplicating data are the same process. For administrators, this makes all the difference, as unlike post-process deduplication, with inline deduplication there is no need to manage different pools of storage for landing native data, no need to predict when deduplication will finish and replication will start, or where data will be recovered from, and no need to provision extra storage as a hedge against the inherent unpredictability of the post-process. Of course, vendors will try to hide these details and spin them into features and benefits (until you read the manual), but from what I observed, the customers eyes are now wide open.