Continuous Testing with ElectricCommander

Continuous Testing
with ElectricCommander
© Electric Cloud, Inc. | 2009Continuous Testing with ElectricCommander
Drive quality earlier in the development process with continuous testing.
Large development organizations have increasingly been adopting select agile practices to enable faster development, create more reliable software, and reduce costs. Chief among these practices is providing developers early and frequent feedback from users, from other developers, and especially from the programming, build, and test tools. In the latter category, continuous integration and continuous testing are emerging as valuable techniques for identifying errors early so they can be corrected immediately by the developer who made the code changes.
Most organizations that want to implement these practices face the challenge of scalability-how to implement continuous integration and testing for code bases that contain tens of millions of lines of code, involve many teams of developers, or run on numerous targets and platforms.
Electric Cloud specializes in tools purpose-built for projects of true enterprise scale. This white paper discusses ElectricCommander and how it uniquely delivers enterprise-level continuous integration with full support for all forms of continuous testing-without requiring modification to existing infrastructure, test tools, or established practices.
The Benefits of Early Defect DetectionIt is a well-established axiom in software development that the later a defect is detected, the more expensive it is to fix. Many sources claim, and many managers confirm, that the cost of remediation jumps exponentially as the software moves from one phase to the next: from requirements and design to coding, testing, and production. Research suggests the costs of defect repair are 1x during requirements and design, more than 10x during coding and testing, and 100x after release to production.
1These numbers, which are sometimes referred to as the 1-10-100 rule, probably understate the cost of fixing errors postproduction, because they do not reflect the unseen costs of lost productivity nor the lost sales resulting from customer-visible defects.
The high cost of defects is one of the main drivers of the agile development movement, which holds that development organizations should release frequently, solicit constant feedback, and easily accommodate changes initiated by bug discovery or modifications of customer requirements. One of the key principles underlying this agile approach is "test early, test often."
Continuous IntegrationA leading best practice in support of this "test often" dictum is one that has gained considerable acceptance during the last few years: continuous integration. In its most basic form, continuous integration, or CI, rebuilds the software and runs unit tests every time code is checked into the main repository. The goal is to identify, as early as possible, coding problems that will "break" the build; that is, prevent it from completing or running satisfactorily. Build-breaking problems can then be resolved quickly and development can continue apace.
2.Good CI software can determine whose code broke the build. If the CI server can identify the problem quickly, then the corrective steps can be taken while the developer is in immediate touch with the code changes and the rationale behind them. If, instead, the defective code is found a day or a week later, the developer is likely to be working on something else, and re-entering the defective code to correct the error will involve a longer ramp-up. In addition, by not having all the details at his fingertips, the developer is more likely to make changes that break other code or that lead to unexpected repercussions. This cost is compounded by the likelihood that other code has been layered on top of the defect, and so fixing the defective code can result in expensive changes to other modules and tests.
CI implementations can run into problems of scale, however. Sites that have large code bases, large teams, or multi platform environments can find it difficult to run the build cycle sufficiently fast enough for the CI server to validate builds multiple times during the workday. This scenario is particularly true at sites where developers check in code frequently. Several means of addressing this issue are available today. The first is to use a build engine that can ... [download for more]