Optimizing new IBM servers with Intel Xeon 5500 processors

How memory configuration can affect the performance of System x and BladeCenter servers March 2009
Optimizing the Performance of
IBM System x and BladeCenter
Servers using Intel Xeon 5500
Series Processors Ganesh Balakrishnan IBM System x and BladeCenter Performance Ralph M. Begun IBM System x Development Optimizing the Performance of IBM System x and BladeCenter Servers using Intel Xeon 5500 Series Processors Page 2
1.0 Introduction ® ®The Intel Xeon 5500 series processors are next-generation quad-core processors targeted at the two-socket server space and will be the common building block across a number of IBM ® ®platforms, including the IBM BladeCenter HS22 blade server, the 1U x3550 M2 and 2U x3650 TM2 rack servers, and the IBM iDataPlex dx360 M2 server. With the Xeon 5500 series processors, Intel has diverged from its traditional Symmetric Multiprocessing (SMP) architecture to a Non-Uniform Memory Access (NUMA) architecture. In a two-processor scenario, the Xeon 5500 series processors are connected through a serial coherency link called QuickPath Interconnect (QPI). The QPI is capable of 6.4, 5.6 or 4.8 GT/s (gigatransfers per second), depending on the processor model. The Xeon 5500 series integrates the memory controller within the processor, resulting in two memory controllers in a two-socket system. Each memory controller has three memory channels and supports DDR-3 memory. Depending on processor model, the type of memory used, and the population of memory, memory may be clocked at 1333MHz, 1066MHz or 800MHz. Each memory channel supports up to 3 DIMMs per channel (DPC), for a theoretical maximum of 9 DIMMs per processor or 18 per 2-socket server. (See Figure 1 for illustration.) However, the actual maximum number of DIMMs per system is dependent upon the system design.
Xeon 5500 Xeon 5500 Processor 0 Processor 1 QPI Memory Controller Memory Controller
Ch0 Ch1 Ch2 Ch0 Ch1 Ch2

Figure 1. Xeon 5500 architecture showing maximum memory capabilities
2.0 System Architecture ®In this section, we will explore the system architectures of various IBM System x and BladeCenter servers equipped with Xeon 5500 series processors, from a memory standpoint. 2.1 HS22 Blade HS22 is designed with 12 DIMM slots as shown in Figure 2 and Figure 3. The 12-DIMM layout provides 6 DIMMs per socket and 2 DIMMs per channel (DPC). Optimizing the Performance of IBM System x and BladeCenter Servers using Intel Xeon 5500 Series Processors Page 3
Xeon 5500 Xeon 5500 Processor 0 Processor 1
QPI Memory Controller Memory Controller
Ch0 Ch1 Ch2 Ch0 Ch1 Ch2
1 5 3 7 11 9
2 6 4 8 12 10
Figure 2. HS22 DIMM slots architectural layout
Processor 1
7 6
8 5
FRONT 9 4 of Blade BACK Server 10 3
11 2
12 1
Processor 0 Figure 3. HS22 DIMM slots physical layout
2.2 System x3550 M2, x3650 M2, and iDataPlex dx360 M2 As shown in Figure 4 and Figure 5 below, the other IBM servers containing Xeon 5500 series processors-the System x3550 M2, the x3650 M2, and the iDataPlex dx360 M2-each provide 16 DIMM slots. Like the HS22, each processor has an equal number of DIMM slots. However, unlike the HS22 all memory channels do not have equal DPC (DIMMs per channel). Optimizing the Performance of IBM System x and BladeCenter Servers using Intel Xeon 5500 Series Processors Page 4
Xeon 5500 Xeon 5500 Processor 0 Processor 1
QPI Memory Controller Memory Controller
Ch0 Ch1 Ch2 Ch0 Ch1 Ch2
1 4 7 9 12 15
2 5 8 10 13 16
3 6 11 14 Figure 4. x3550 M2/x3650 M2/dx360 M2 DIMM slots architectural layout
Processor 1 8 9 7 10 6 11 5 FRONT of Chassis 12 BACK 4 13 3 14 2 15 1 16 Processor 0 Figure 5. x3550 M2/x3650 M2/dx360 M2 DIMM slots physical layout
3.0 Memory Performance With the varied number of configurations possible in the Xeon 5500 series processor-based systems, a number of variables emerge that influence processor/memory performance. The main variables are memory speed, memory interleaving, memory ranks and memory population across various memory channels and processors. Depending on the processor model and number of DIMMs, the performance of the Xeon 5500 platform will... [download for more]