Test Process Assessments Move into the Real World

A Rethinking QA White Paper

Test Process Assessments
Move into the Real World © Lionbridge 2009
White Paper | Copyright Lionbridge 2009 | WP-540-0609-1| Page 1 White Paper
At a Glance Freed from the classroom, frameworks for measuring the effectiveness of test organizations have been adapted to the real world, and are in use today by development teams looking to continuously improve product release readiness.
Executive summary Every development organization has a test and quality assurance process -- whether it is an efficient one is a different story. A test team may be saddled with outdated processes from a previous generation of products, lacking a mechanism that encourages continuous improvements in test methodology. Or, a comprehensive QA roadmap may be absent altogether. Previous efforts to systematically measure the efficiency and maturity of a QA/test organization have been largely academic exercises, difficult to apply to real world software development environments. Today, however, new methods are available that enable product developers to assess and improve their quality assurance and test processes, without interfering with day-to-day production 1demands, using well established frameworks such as the Testing Maturity Model index. This white paper examines the fresh techniques for assessing the effectiveness of QA organizations. Whether an organization has a dedicated testing team or not, today's testing is being completed more and more often by product managers, business analysts, customers, and programmers in addition to traditional testers. Evaluating all of these testing practices through a unified lens, using a maturity scoring scale that looks at all key testing responsibilities and functions, can be accomplished with the application of tried and true assessment frameworks. Then a record of QA/testing strengths and weaknesses can be drafted with a proscribed improvement plan that has buy-in from all product development stakeholders.
What is test process assessment? Test process assessment is a structured evaluation of existing testing practices and associated documentation and tools within a development organization. The assessment involves bottom-to-top interviews with product development and test team members, from entry-level test technicians to Engineering VPs. Artifacts reviewed by assessors may include test planning and strategy documents, quality policies, test tools and frameworks, and individual test cases used on recent projects. Since testing activities are often deeply embedded in many facets of the software development lifecycle (SDLC), a thorough analysis of testing practices involves examination of other development activities such as requirement specification and release management.
1 Developed by Dr. Ilene Burnstein, et al, at the Illinois Institute of Technology in 1996
White Paper | Copyright Lionbridge 2009 | WP-540-0609-1| Page 2
The assessment exercise provides a test maturity baseline of the current state of testing practices, one that can be used as a reference for future evaluations, after suggested improvements have had a chance to work.
Why consider test process assessments? There are several drivers that lead organizations to consider test process assessments. Some developers are motivated by past product quality issues resulting from high escaped-defect rates, and the subsequent customer complaints and poor product adoption patterns. Others are driven by time-to-market pressures; as inefficient testing is often the bottleneck that delays release schedules. A poor record of meeting service and quality level agreements may also prompt development teams to launch across-the-board QA process improvement initiatives. Earning certifications, awards, or industry recognition are also typical reasons to consider following an improvement model.
How can test assessment and improvement models help? Assessment and improvement models provide a framework for planning and conducting appraisals and defining a manageable and traceable improvement plan. They ensure the right questions are asked of the right stakeholders for each specific area under evaluation. The models guide users in recognizing and prioritizing processes that need improve... [download for more]