Storage Infrastructure Utilization:
Endless capital requests for additional storage resources are the bane of existence for many storage directors and infrastructure VPs. The seemingly unquenchable demand for more space, combined with the nagging feeling that existing space is not being used, has triggered many an internal project aimed at examining storage utilization issues. All too often, these projects attempt to chase the metric of file level utilization. Yet unfortunately, this may not be the right approach to solving utilization problems. Especially for large, complex storage environments, attempting to track file utilization on a global basis may be doomed to failure. This white paper proposes a more effective approach to solving the storage infrastructure utilization challenge in complex SANs.
Why storage infrastructure underutilization?
What leads application owners to over request storage? Through experience helping corporations gain control over networked storage assets, Onaro has observed the following reasons:
- Fear of the 2:00 AM page. It only takes one experience of being paged in the middle of the night due to insufficient database storage for a DBA to forever over request space.
- Uncertainty of actual demand ? Many application teams do not have a clear idea of exactly how much space their application will require. Factor in transient storage demands and the picture gets cloudier.
- Lack of a service level for provisioning time Without an agreed upon time frame to provision new storage, application owners are uncomfortable with an open ended schedule for getting new storage in a crisis. Without this service level, storage provisioning times are uneven at best. To overcompensate for this, teams over request to compensate for the unpredictable lead time.
- Financial incentive to conservatively estimate storage requirements is outweighed by career disincentive of running out of space. Without a simple cost allocation system, it is always easier to overrequest space.
- Space request multiplication. Space request formulas start with the application team requesting "X" amount of space. Then the system administrator doubles that amount to avoid being awakened in the middle of the night due to insufficient storage. Finally, the storage team adds another 20-30% to prevent having to scramble to add more space due to the application team's underestimation of requirements.
- Uncertainty over actual loads in the SAN fabric and arrays. Without the ability to understand exactly how applications are loading the SAN fabric and arrays, many storage teams hesitate to push the envelope on both port and array utilization. The result is underutilization of switch ports and arrays that are not fully allocated.
The False Promise of FileLevel Utilization
To solve the storage infrastructure utilization challenge, many organizations are attempting to monitor file level utilization. In Onaro's opinion, once the size of a SAN exceeds a few hundred ports, the time, effort and cost of tracking file level utilization across the entire environment makes this an impractical and fruitless exercise that fails to solve the core utilization problems.
Why? First, most of the utilization problem is a human behavior and operations challenge. Having reports on filel evel utilization does not address the underlying reasons for overestimating storage requirements i.e., the 2:00 AM page problem, the uncertainty of demand, the slowness of provisioning new space, the lack of financial incentive, or space request multiplication. What filelevel storage infrastructure utilization does provide is an early warning before a system runs out of space and a feedback mechanism that helps calibrate future storage requests. But what is the cost of globally tracking filelevel utilization?
Unfortunately, understanding file utilization requires the deployment of operating system agents, along with the supporting infrastructure to manage and interrogate these agents. In the majority of organizations with whom Onaro has worked, the ability to successfully deploy agents drops off dramatically once the host count rises to about 7080.
The complexity of maintaining compatible agents, operating systems, and agent control applications combined with the crossfunctional demands of coordinating agent deployments makes this a task fit for Sisyphus. In a typical scenario observed by Onaro, an organization with more than 10,000 ports had several fulltime administrators just working on agent deployment. It took them over 18 months to complete a full agent roll out, and at any given point in time a large percentage of agents were not reporting. The result was a constant inability to accurately report on storage resource consumption by application.
