Diligent HyperFactor: Redefining De-Duplication

T E C H N O L O G Y B R I E F
Diligent HyperFactor: Redefining De-Duplication August 2006 In the past 24 months, we have seen disk-based data protection and archiving become an essential component of most enterprises. With massive 100 TB+ disk-based repositories becoming increasingly common on near-term enterprise roadmaps, finding ways to optimize those disk capacities is now a top of mind concern. Even with widely available cost-effective disk, our research finds that capacities still account for over 50% of most data storage budgets, with disk-based secondary storage very commonly exceeding 50% year-over-year growth rates. Aggressively managing these capacities to keep them "lean and mean" is absolutely critical. The good news is that new technologies exist that can radically slim down the physical storage requirements for secondary disk storage. Taneja Group refers to these technologies as Capacity Optimization (CO) technologies. Every enterprise now needs a CO strategy, but beware, for not all solutions are created equal. In our opinion, one company on the very cutting edge of this CO wave is Diligent Technologies. Specifically, we call attention to Diligent's HyperFactor technology. It stands out as extremely scalable, efficient, high-performance data de-duplication software that merits a deeper understanding. As the "secret sauce" in Diligent's ProtecTier VTL solution, HyperFactor has achieved efficiencies that represent a true leap over 1st generation optimization approaches. Beyond the mind-bending 25:1 reduction ratio, the efficiency of this technology's indexing architecture (and therefore, its performance) is second to none. In this technology brief, we will examine what IT teams should be looking for in a capacity optimization solution, and then share our perspective on the Diligent HyperFactor technology. This is a core new technology and we're very excited by what it can mean for storage ROI, data protection performance, and reliability. Why Optimize Storage Capacity? This entire CO category has gone from marginal to essential in the past 24 months. Somewhat ironically, the latest growth boom Why the intense interest in optimizing in the storage industry is all about reducing storage capacities? Two words: economics capacities. Specifically, this innovation boom and manageability. IT teams need to reduce is about software technologies that promise the amount they are spending to store a given to radically reduce the amount of physical terabyte of information, and they need to storage capacity required to store a given find ways to manage the insanely high amount of information. In fact, some of the growth rates common across the enterprise most exciting companies in storage today are today. Note that the biggest culprits of laser-focused on tackling precisely this capacity consumption have been data challenge. Taneja Group has categorized all protection and archiving environments of these offerings under the umbrella of (collectively, "secondary storage".) For that Capacity Optimization (CO) technologies. reason, significant vendor R&D attention has Copyright© The TANEJA Group, Inc. 2003. All Rights Reserved 1 of 887 Elm Street, Suite 900 Hopkinton, MA 01748 Tel: 508-435-5040 Fax: 508-435-1530 www.tanejagroup.com
T E C H N O L O G Y B R I E F
been focused on finding ways to optimize fine-grained the granularity, the more those capacities in order to improve storage efficient the entire repository can become. All ROI and ease of management for large of this requires an ongoing, dynamic dialog repositories. between the repository and the index, as every new piece of data created immediately Indeed, these CO technologies have already changes what will need to be stored in the become very powerful. As we will explore future. below, the most advanced offerings can achieve reduction ratios of 25:1 or more on From this high-level description, it should disk-based storage platforms, scaling into the already be obvious that the power of any petabyte range for single solutions, all with given de-duplication technology rests no impact to the pre-existing data pr... [download for more]