Test Estimation Challenges 1. a.
Successful test estimation is a challenge for for most organizations coz No standard Formulae/Methods for Test Estimation
b. Test Effort Estimates also also includes the Debugging Effort c. Difficult to attempt attempt testing estimates without first first having detailed information information about a project d. Software Testing Testing Myth that that Testing Testing can be performed performed at the End e. Difficult to attempt attempt testing testing estimates estimates without an understanding of what should should be included in a 'testing' estimation for a project (functional testing? u nit testing? reviews? inspections? load testing? security testing?) Traditional Practices 1. a. i. ii. b. i. ii. c. i.
Current Test Planning Methods include Using Percentage of Development Effort Depends on the accuracy of the Development Effort Does not account revisit of Development Effort Using Tester Developer Ratio May not be same for all types of Projects Does not consider the size of the Project Using KLOC Does not consider consider Complexity, Criticality & Priority of the Project
What’s the Best Approach to Test Estimation There is no simple answer for this. The 'best approach' is highly dependent on the particular organization and project and the experience of the personnel involved. For example, given two software projects of similar complexity and size, the appropriate test effort for one project might be very large if it was for life-critical medical equipment software, but might be much smaller for the other project if it was for a low-cost computer game. A test estimation approach that only considered size and complexity might be appropriate for one project but not for the other. Approaches to Test Estimation Implicit Risk Context Approach: Metrics-Based Approach: Test Work Breakdown Approach: Iterative Approach:
Percentage-of-Development Approach: Test Estimation Process – A Practical Approach 1. 2. 3. 4. 5.
Combination of all the Approaches Considers Risk & Complexity Factors Based on Previous History i.e. Organization or Project Metrics Based on Work Breakdown Structure Based on Iterative Model
Elements of Test Estimation Process 1. Break sizing into smaller, easier to estimate tasks a. Decompose the test project into phases. i. System Test ii. Unit Test b. Decompose each phase into constituent activities i. System Test Planning ii. Test Execution c. .Decompose each activity into tasks and subtasks until each task or subtask at the lowest level of composition i. Executing a test scenario ii. Writing a defect 2. Taking risk priority into account 3. Set up dependencies a. Dependent tasks internal to the test subproject. b. Document dependencies, resources, and tasks external to the test subproject (i.e., those that involve collaborative processes) 1. Consider type of code (complex, reused, etc.) 2. Augment professional judgment and gut instinct with previous project data, industry metrics, and so forth. 3. Identify and, if possible, resolve discrepancies between the test subproject schedule and the project schedule. 4. Use the work-breakdown-structure and schedule to develop a budget. Extract from your work-breakdown-structure a complete list of resources. For each resource, determine the first and last day of assignment to the project. 5. If you have resources shared across multiple test projects within a given time period, understand the percentage allocation of each resource’s assignment to each project during various time periods. 6. Revisit the Estimation continuously in order to reflect any change in the Project Requirements or Schedule 7. Be Repeatable preferably Automat able A Sample Estimation Process
Total work = Test case time + Defect time Test Case Time = Test Case Development time + Test Case Execution Time Defect Time = (Hours/Defect * # Defects) Note: Consider the defect severities while arriving at Hours/Defect and also that Hours/Defect should take into account the Defect creation time, Debugging time and Defect Retesting time till the closure. Test Case Time Test Case Development Time = (Hours/Test case development* #Test cases) Test Case Execution Time = (Hours/Test case Execution * #Test Cases) Note: Consider the Risk, Complexity while arriving at the Hours/Test case Development and Hours/Test case Execution
Software Test Estimation - 9 General Tips on How to Estimate Testing Time Accurately Posted In | Testing Life cycle, Test strategy
This is a guest article by Author “N. Sandhya Rani”. For success of any project test estimation and proper execution is equally important as the development cycle. Sticking to the estimation is very important to build good reputation with the client. Experience play major role in estimating “software testing efforts”. Working on v aried projects helps to prepare an accurate estimation for the testing cycle. Obviously one cannot just blindly put some number of days for any testing task. Test estimation should be realistic and accurate. In this article I am trying to put some points in a very simple manner, which are helpful to prepare good test estimations. I am not going to discuss the standard methods for test estimations like testing metrics, instead I am putting some tips on - How to estimate testing efforts for any testing task, which I learned from my e xperience. Factors Affecting Software Test Estimation, and General Tips to Estimate Accurately: 1) Think of Some Buffer Time
The estimation should include some buffer. But d o not add a buffer, which is not realistic. Having a buffer in the estimation enables to cope for any delays that may occur. Having a buffer also helps to ensure maximum test coverage. 2) Consider the Bug Cycle The test estimation also includes the bug cycle. The actual test cycle may take more days than estimated. To avoid this, we should consider the fact that test cycle depend s on the stability of the build. If the build is not stable, then developers may need more time to fix and obviously the testing cycle gets extended automatically. 3) Availability of All the Resources for Estimated Period The test estimation should consider all the leaves planned by the team members (typically long leaves) in the next few weeks or next few months. This will ensure that the estimations are realistic. The estimation should consider some fixed number of resources for test cycle. If the number of resources reduces then the estimation should be re-visited and updated accordingly. 4) Can We Do Parallel Testing? Do you have some previous versions of same product so that you can compare the output? If yes, then this can make your testing task bit easier. You should think the estimation based on your product version. 5) Estimations Can Go Wrong - So re-visit the estimations frequently in initial stages before you commit it. In early stages, we should frequently re-visit the test estimations and make modification if needed. We should not extend the estimation once we freeze it, unless there are major changes in requirement. 6) Think of Your Past Experience to Make Judgments! Experiences from past projects play a vital role while preparing the time estimates. We can try to avoid all the difficulties or issues that were faced in past projects. We can analyze how the previous estimates were and how much they helped to deliver product on time. 7) Consider the Scope of Project Know what is the end objective of the project and list of all final deliverables. Factors to be considered for small and large projects differ a lot. Large project, typically include setting up test bed, generating test data, test scripts etc. Hence the estimations should be based on all these factors. Whereas in small projects, typically the test cycle include test cases writing, execution and regression. 8 ) Are You Going to Perform Load Testing? If you need to put considerable time on performance testing then estimate accordingly. Estimations for projects, which involve load testing, should be c onsidered differently. 9) Do You Know Your Team?
If you know strengths and weaknesses of individuals working in your team then you can estimate testing tasks more precisely. While estimating one should consider the fact that all resources may not yield same productivity level. Some people can execute faster compared to others. Though this is not a major factor but it adds up to the total delay in deliverables. And finally tip number 10. Over To You! This test estimation tip is purposefully left blank so that you can comment your best estimation techniques in below comment section.
