Rapid Bottleneck Identification - A Better Way to do Load Testing

Rapid Bottleneck Identification-
A Better Way to do Load Testing
An Oracle White Paper June 2009
Rapid Bottleneck Identification-
A Better Way to do Load Testing

INTRODUCTION You're ready to launch a critical Web application. Ensuring good application performance is crucial, but time is short. How can you optimally test the application and still meet your deadlines?
Rapid bottleneck identification (RBI) is a new testing methodology that allows . quality assurance (QA) professionals to very quickly uncover Web application RBI combines a comprehensive performance limitations and determine the impact of those limitations on the end understanding of bottlenecks with a refined user experience. Developed through years of testing engagements across all types load testing methodology that enables of platforms, the RBI methodology dramatically reduces load testing cycles while organizations to create highly scalable Web allowing more-and more thorough-testing. Using this approach, organizations applications. can improve application quality, enhance the customer experience, and lower the cost of deploying new systems.
PERFORMANCE TESTING DEFINED Performance testing can be roughly defined as "testing conducted to evaluate the compliance of a system or component with specified performance requirements." However, every application has at least one bottleneck, and few, if any, systems ever meet initial performance requirements. To reflect this reality, let's redefine performance testing as "testing conducted to isolate and identify the system and application issues (bottlenecks) that will keep the application from scaling to meet its performance requirements."
This philosophical shift in perspective-from testing as an evaluation to testing as an active investigation to isolate and resolve problems-is what drove the creation of the RBI methodology. RBI combines a comprehensive understanding of bottlenecks with a refined testing methodology that enables organizations to create highly scalable Web applications.

Rapid Bottleneck Identification-A Better Way to do Load Testing Page 2 UNDERSTANDING BOTTLENECKS, THROUGHPUT, AND CONCURRENCY Before delving into the specifics of the RBI methodology, we must first establish a common understanding of bottlenecks-and where they are found-as well as draw a distinction between throughput and concurrency testing.
Bottlenecks-Key Performance Inhibitors Any system resource-such as hardware, software, or bandwidth-that places defining limits on data flow or processing speed creates a bottleneck. In Web applications, bottlenecks directly affect performance and scalability by limiting the amount of data throughput or restricting the number of application connections. These problems occur at all levels of the system architecture, including the network layer, the Web server, the application server, and the database server. Historically, based on our experience testing actual customer applications, bottlenecks have been distributed across these components as shown in Figure 1.

In Web applications, bottlenecks directly affect performance and scalability by limiting the amount of data throughput or restricting the number of application connections.
Figure 1. Estimated distribution of bottlenecks across the system infrastructure.
The Compounding Impact of Testing Complexity The testing approach you choose directly impacts the difficulty of isolating and resolving bottlenecks. Unfortunately, too many testing procedures begin with complex usage scenarios where testers try to simulate exactly how the application will be utilized in production. This may involve running several different transactions to simulate different types of users who interact with the application in different ways. Unfortunately, this creates a significant testing roadblock because scenarios that are higher in complexity and involve multiple different transactions introduce more bottlenecks into the test, which makes it difficult to identify root causes.
For example, the graph in Figure 2 illustrates the test results of a standard e-commerce application that bottlenecked at approximately 2,000 concurrent users. In this sample test, the usage scenarios involved browsing, searching, and adding items to a shopping cart to complete a purchase. Although there were o... [download for more]