This white paper primarily focuses on the expanding role of HPC management software in the HPC marketplace. The paper emphasizes its historical development and considers its business case and future prospects while dispensing with more elaborate technical analysis and product comparisons.
W H I T E P AP E R H P C M a n a g e m e n t S o f t w a r e : R e d u c i n g t h e C o m p l e x i t y o f H P C C l u s t e r a n d G r i d R e s o u r c e s Sponsored by: Platform Computing Steve Conway Richard Walsh Earl C. Joseph, Ph.D. May 2008 moc.cd I D C O P I N I O N i.ww The typical architecture of a high-performance computing (HPC) resource is no longer w a single custom-engineered, integrated hardware and software system with 510 collections of vendor-ported and tested applications. Today, HPC resources are 4.5 typically constructed à la carte from standards-based hardware and software 39.8 component technologies. These resources are likely to include multiple systems at 05. multiple sites. The variety and number of those components, their distinct providers, F their potentially different locations, and the costliness of testing user applications 002 against them all have driven up HPC resource management complexity and created 8.2 an integration gap. Most HPC system configurations purchased today are unique 78.8 assemblages of hardware and software. The growing task of integrating them into a 05. reliable, productive, and secure working environment has fallen heavily on system P administration and user support personnel. ASU 1 The careful design, selection, and integration of the software components between 071 the operating system (OS) and application layers of the software stack are 0 A increasingly vital for reducing the complexity of HPC cluster and grid management M , and for presenting an integrated and responsive HPC resource to users. Often mah informally referred to as middleware, the number of components in midstack HPC gni management software has grown significantly in the past 10 years. In IDC's opinion, mar this process has exposed a growing opportunity for vendors to add value to the HPC F t work cycle by adopting, streamlining, and modularizing the now long list of software eert functions assumed by HPC management software, including: S nee ! p User application compilation, debugging, and profiling S 5 : ! s User and system application input/output translation and scripting retra ! u User application libraries, mathematical, IO, and parallel (e.g., MPI, OpenMP) qdae ! Workload queuing, scheduling, and management (e.g., migration, checkpoint, H la restart) bolG ! System and application installation, integration, and patching ! System management — servers, software, security, disk storage, and backups ! System and application monitoring, reporting, provisioning, reconfiguration, and failure detection
M E T H O D O L O G Y
The sources for this white paper on the origins, recent developments, and growing importance of HPC management software (often informally referred to with the catch-all term middleware) in the HPC market include recent IDC research studies and IDC-sponsored HPC conferences, as well as discussions with HPC users, buyers, and hardware and software vendors (including Platform Computing). In addition, IDC conducted interviews with Platform Computing customers (Citigroup, Advanced Micro Devices [AMD], and ARM Holdings) and a Platform Computing business partner (SAS) to benchmark IDC's thinking against opinions in the field.
I N T H I S W H I T E P AP E R
This white paper primarily focuses on the expanding role of HPC management software in the HPC marketplace. The paper emphasizes its historical development and considers its business case and future prospects while dispensing with more elaborate technical analysis and product comparisons.
This white paper reviews the rise of clusters to dominate the HPC market and points out the elevating effect it has had on the role of the software in use between the operating system and application layers. It clarifies these midstack boundaries and suggests what the HPC management software variety of middleware is and what it is not. It points out how the HPC cluster revolution has set this part of the software stack on a path to become an essential, global binding agent for today's sprawling HPC hardware and software infrastructure. The paper discusses the challenges associated with managing clusters and grids and the ... [download for more]
