|
Service-oriented architecture (SOA) promises to shorten development cycles, increase re-use, and make enterprises better able to respond to business change. And it does. But SOA requires IT professionals at all levels to rethink what they do and how they do it. Dr. Ferhan Kilical, now test manager at Northrop Grumman Mission Systems, has been managing the testing of large-scale, mission-critical systems built on SOA for over five years. “SOA works!” she says. “But it changes everything for the quality organization.” Managing IT quality has always been a hard job.
There are too many applications and not enough time and resources. Test effort must be allocated based on the risk presented by each module. In SOA, re-usable services can amplify risk—a failure in a critical service can ripple through all the applications and other services that access it. And SOA means more things to test and more lifecycles to manage. Traditional test tools—designed to test application transactions— provide little help in testing services with broad functionality and no user interface. Dr. Kilical uses HP Quality Center software and HP Performance Center software to implement a risk-based testing program for SOA that uses a lifecycle approach to track quality from requirements through testing to release. This paper describes the challenges Dr. Kilical and other IT quality managers face when applying traditional test tools and processes to SOA, and it shows how HP has adapted HP Quality Center and HP Performance Center software to help IT meet the new needs of SOA.
SOA: the quality challenge
Top-notch test teams plan the test effort to be applied to each facet of the application, use automated tools to perform tests efficiently, and analyze results to make release decisions. But SOA—inherently distributed, loosely coupled, asynchronous, and heterogeneous— brings new challenges to established processes.
Dr. Kilical, a Six Sigma Green Belt, speaks in the breathless flurry of a harried IT manager juggling dozens of projects. She describes a complex environment built on technologies often new to the test organization and presenting a level of generality that makes it difficult to know where to apply the most test effort. “There are many more lifecycles to manage in SOA. We have to test each service as it’s developed, then we have to test the integrations between services and the applications that use them. And some services have no GUI, so our traditional test tools didn’t help at all.”
Volume and complexity
It is the complexity that presents much of the challenge. SOA requires the development organization to build the services, test them, and then build applications that access them. And when services change—and they change often—regression testing must validate that the other services and applications that access them still work. As a result, the number of independent lifecycles to be managed grows exponentially, especially when the organization adopts agile development methods that roll out functionality in iterations. Test teams need fast, efficient test processes to handle the volume of testing generated in a SOA. And the test organization needs a way to keep track of which applications and services use which other services so regression testing can be done when needed.
And there is more to be tested—the sheer volume of testing required is dramatically increased. In each test cycle, testing must confirm not only that services and applications work as specified but that they conform to functionality, performance, and security policies established by SOA governance. Policies may dictate certain mandatory methods for objects, or specify how they must handle exceptions. Security policies are carefully specified for every service. Conformance to all of these policies must be verified.
Services and applications may be built by different development teams using different development environments and toolkits, so interoperability testing is critical. The Web Services Interoperability Organization (WS-I) has established profiles to help assure interoperability. Those tests must be run before the service can claim WS-I compliance. “We have teams using Java™ and other development tools. And some services are provided by third parties who may use something different still. It’s supposed to work, but we have to test it to be sure.” Finally, because SOA is loosely coupled, there are more boundary conditions to test. All this means more lifecycles and more testing per lifecycle.
|
