Test Plan P lan Document D ocument INTERNATIONAL-KIDS.COM DEVELOPMENT PROJECT
Prepared by Netizen Team Version 1.0 Created on : 10-Oct-2007 Last Modified : Document: Test Plan
Page 1 of 35
INTERNATIONALKIDS.COM Revision History Versio n/ Revisio n Numbe r
Author
1.0
Netizen
Description
Approve r
Effective Date
Initial Test Plan Draft
Page 2 of 35
INTERNATIONALKIDS.COM Table of Contents 1 INTR INTRODUC ODUCTION TION........ ................ ................ ................ ................ ................ ................. ................. ................ ................ ................ ................. ................. .......... .. ............... .......... 4
1.1 1.1 1.2 1.2 1.3 1.3
Purpose of this Document......................................................................................................4 Purpose of this Document......................................................................................................4 Overview..................................................................................................................................4 Overview..................................................................................................................................4 Scope........................................................................................................................................5 Scope........................................................................................................................................5
1.3.1 Testing Phases...................................................................................................... .........................5 1.3.2 Testing Types.......................... ...................... ....................... ....................... ...................... .......... ....5
1.4 Not in Scope............................................................................................... ...........................7 .......................... .7 1.4 Not in Scope............................................................................................... ...........................7 .......................... .7 1.5 Reference Documents.............................................................................................................7 1.5 Reference Documents.............................................................................................................7 1.6 Definitions Definitions and Acronyms........................................................................................... Acronyms........................................................................................... ...........7 ........ ...7 1.6 Definitions Definitions and Acronyms........................................................................................... Acronyms........................................................................................... ...........7 ........ ...7 1.17 Assumptions Assumptions and Dependencies....................................................... Dependencies......................................................................................... .................................. ..7 1.17 Assumptions Assumptions and Dependencies....................................................... Dependencies......................................................................................... .................................. ..7 1 TEST REQUIREMENT.................... ........................................... ............................................. ............................................. .............................................. ........................... .... ..... ......... .... 8
2.1 Features Features to be Tested................................................................................... .............. ....... .............8 ......8 2.1 Features Features to be Tested................................................................................... .............. ....... .............8 ......8 Milestones (Schedule)..................................................................................................................9 Milestones (Schedule)..................................................................................................................9 2 TESTING ENVIRONMENT.................... .......................................... ............................................. ............................................. ............................... ......... .... ......... ......... ......... ..... 10
1.18 1.18 1.19 1.19
Browsers.............................................................................................................................10 Browsers.............................................................................................................................10 Hardware Hardware and Software Software Requireme Requirements nts ................................................................... ............................................................ .............. .......10 10 Hardware Hardware and Software Software Requireme Requirements nts ................................................................... ............................................................ .............. .......10 10
3.2.1 Offshore...................... Offshore...................... ....................... ...................... ....................... ....................... .... .......... ..... ......... ........10 ....10 1.1.1.1 Development/ Development/ Development Integration Environment: Environment:...................... .......................................... .................... .... ......... ......... ...... ..10 10 1.1.1.2 QA....................... ............................................. ............................................. ............................................. ............................................. ......................................... .................. .... ...... .. 11
1.20 Human Resources...............................................................................................................12 1.20 Human Resources...............................................................................................................12 3 ROLES AND RESPONSIBILITIES...................... ............................................ ............................................. ................................. .......... .... ......... .......... .......... ......... ...... .. 12 4 TEST STRA STRATEGY TEGY.................... ........................................... .............................................. ............................................. ............................................. ...................................... ............... ..... 13
Test Process Workflow..............................................................................................................14 Test Process Workflow..............................................................................................................14 1.21 Test Test Organize/Review Organize/Review Project Documentation Documentation..................................... ............................................ .............. .............. .............15 ......15 1.21 Test Test Organize/Review Organize/Review Project Documentation Documentation..................................... ............................................ .............. .............. .............15 ......15 1.22 Develop System Test Test Plan........................................................................ ...........................15
Page 3 of 35
INTERNATIONALKIDS.COM 1.22 Develop System Test Test Plan........................................................................ ...........................15 1.23 Test Design/Development...................................................................................................15 1.23 Test Design/Development...................................................................................................15 1.24 Unit Test Test Execution............................................... Execution............................................... ........................................................ ..............................................................17 ......17 1.24 Unit Test Test Execution............................................... Execution............................................... ........................................................ ..............................................................17 ......17 1.25 Integration/System Integration/System Test Test Execution ................................................................................. ............................ .......................................................17 ..17 1.25 Integration/System Integration/System Test Test Execution ................................................................................. ............................ .......................................................17 ..17 1.25.1Integration 1.25.1Integration Testing.................................. ...................... ....................... ....................... ..... .......... ...... .....17 .17 1.25.2System Testing.................................. ...................... ....................... ....................... ................... ....18 .... 18 5.6.3 Testing Types ....................... ...................... ....................... ...................... ....................... ......... .....18 ..... 18 5.6.4 Test Execution workflow ............................ ..... ....................... ....................... ...................... ............. ... ......... ..... ......... .......... .....22 22
1.26 Defect Tracking Tracking and Management Management ................................................................... .............. ....... .........27 ..27 1.26 Defect Tracking Tracking and Management Management ................................................................... .............. ....... .........27 ..27 1.27 Update Documents Documents and Results......................................................... Results..................................................................................... ................................32 ....32 1.27 Update Documents Documents and Results......................................................... Results..................................................................................... ................................32 ....32 1.28 Test Reports ........................................................................................................................32 1.28 Test Reports ........................................................................................................................32 1.29 UAT and Closure ...............................................................................................................33 1.29 UAT and Closure ...............................................................................................................33 5 CONFIGURA CONFIGURATION TION MANAGEMENT.................... .......................................... ............................................. ................................... ............ ..... .......... .......... ......... ...... ..34 6 DELIVERABLES................... .......................................... .............................................. ............................................. ............................................. ............................. ...... ..... .......... ......... ...... 34 ..
1 1.1 1.1
Introduction Purp Purpos ose e of of tthi his s Doc Docum umen entt
The purpose of this document is to outline the Test Strategy/Approach and the Quality Assurance process for the International-kids.com. This document will establish the Syst Sy stem em test test plan plan for for the the International-kids.com application. application. It will will all allow the the develo developme pment nt team, team, busine business ss analyst analysts, s, and proje project ct managem management ent to coord coordina inate te their their efforts efforts and efficiently efficiently manage the testing testing of the site. The QA process process outlined outlined in this Syst System em Test est Plan Plan will will ensu ensure re that that a qu qual alit ity y International-kids.com application application is deployed successfully and on schedule. The intended audiences for this document are all stakeholders of the Internationalkids.com project.
1.2
Overview
Page 4 of 35
INTERNATIONALKIDS.COM The current International-kids.com is Windows XP based, compatible with Office2002 and and writ writte ten n in PHP, PHP, usin using g MYSQ MYSQL L 5.0 5.0 Serv Server er data databa base se.. International-kids.com expectation with the new application is twofold: 1. Front Office functionalities and 2. Back Office functionalities The focus is primarily on successful migration and implementation o f the application. The main objective of this Test plan is to define define the methodology methodology to test Internationalkids.com application to check and ensure that • New system preserves all of its current business functionalities. • The enhancements have been implemented to the new system. Newe Newerr enh nhan ance ceme ment nts s do not not adve adverrsel sely affe affect ct the the curr urrent ent bu busi sine ness ss • functionalities. • The The system system has flexib flexibili ility ty /capac /capacity ity to deal deal with with comple complex x Internationalkids.com structure and programs, as it continues to change
1.3
1.3. 1.3.1 1
Scope
Testi esting ng Phas Phases es
The following table lists the various phases of o f International-kids.com application testing and the team responsible for it. Phases Unit Testing Integration Testing System Testing User Acceptance Testing
1.3. 1.3.2 2
Teams Responsible
Development team Testing team Testing team Representatives International-kids.com User Representatives
Testi esting ng Types ypes
International-kids.com application will undergo the following types of testing. All types of testing are a re explained in detail under Test Strategy section s ection Activity Functionality Testi sting
Database Testing
Teams Responsible Performed med by testing team during Integ tegratio tion/System testing phase to meet agreed upon functional requirements of International-kids.com application. Follo ollowi wing ng Fun unc ction tional al area areas s will will be put to test test:: (1) (1) Application Submission (2) (2 ) Peer Review. Review. All the features put under test are mentioned in brief under “Features to be tested” section in this plan. All the features put under test will be described in detail in Test Test Scenar Scenario io docume documents nts and Test case case docume documents nts.. On completion completion of every every single single functional functional area, area, test scenario and test case documents will be delivered. Please refer to Deliverables section mentioned below. below. Performed by testing team during Integration/System testing phase to qualify database which houses the content that the International-kids.com application manages, run
Page 5 of 35
INTERNATIONALKIDS.COM Security Testing GUI GUI and and Usab Usabil ilit ity y Testi esting ng Performance and Load /Volume Testing
Code Testing Smoke Testing
Regression Testing
Defect fix verification testing/Defect validation testing) Comp mpa atibility Testing
Interface Testing
•
queries and fulfill user requests for data storage Database migration testing will be taken care by DBA’s Performed by testing team during Integration/System testing phase to meet agreed upon Security requirements of International-kids.com application Perfo erforrmed med by test testin ing g team team du duri ring ng Inte Integr grat atio ion/ n/Sy Syst stem em testing phase Performed Performed by testing team during System Testing Testing phase. Automat Automation ion testing testing will will be perfor performed med to carry carry out these these types of testing. tool will be used to perform these tests. Various Reports that are part of International-kids.com Application will be one of the main areas while performing load/volume testing Performance test methodology. Performed by development team during Unit Testing phase at every method level. Performed by by development te team du during Unit Testing phase for qualifying the build for releasing it to Testing Testing team. Perfor erformed med by Testing esting team team during during Integrat Integration ion/Sy /Syste stem m phase for qualifying the build for further tests. Performed by testing team during Integration/System testing phase for re-testing an entire or partial system after a modification has been made to ensure that no unwanted changes were introduced to the system. Perfo erform rmed ed by test testin ing g team team du duri ring ng Inte Integr grati ation on/S /Sys yste tem m testing phase for verifying the defect fixes Performed med by testing team during Integ tegratio tion/System testing phase to test the compatibility with respect to base configuration (a) Browser Browser (IE 6.0), O.S. O.S. (Win (Win XP) (b) Mozilla fire fox( ), O.S O.S (Win XP) XP) (c) Opera ( ), O.S O.S (Win (Win XP) XP) Cert Certif ific icat atio ion n test testin ing g will will be perf perfor orme med d on foll follow owin ing g combinations : Browser (IE 7.0), O.S (Win XP) Performed Performed by Development team during Unit testing phase Perfo erform rmed ed by test testin ing g team team du duri ring ng Inte Integr grati ation on/S /Sys yste tem m testing phase and Perfor Performed med by International-kids.com team team during during UAT UAT phase in order to have a complete test.
Sign on PA test testin ing g team team will will be respo espons nsib ible le for for test testin ing g this this functionali functionality ty by accessing accessing International-Kids.com -QA environment
Adhoc Testing
Performed by Testing team during Integration/System testing phase to test the (1) Navigations that are unusual
Page 6 of 35
INTERNATIONALKIDS.COM and and (2) (2) Nega Negati tive ve components.
1.4
scen scenar ario ios s
with withiin
and and
acr across oss
the the
Not in Scope 1 2
3
1.5 1.5
Str Stress Testi esting ng Cras Crash/ h/R Recov ecover ery y Testi esting ng When the scope of the new application has been agreed and signed off, no further inclusions will be considered for inclusion in this release, except: Where Where there there is the expr express ess permis permissio sion n and agreem agreement ent of the Busine Business ss • Analyst, Project Manager and the Client; Where the changes/inclusions will not require significant effort on behalf of • the test team (i.e. requiring extra preparation - new test conditions etc.) and will not adversely affect the test schedule.
Refe Refere ren nce Doc Docum umen ents ts
# 1. 2.
Reference Document Name International-Kids.com Software Requirements Document International-Kids.com Test plan
1.6 1.6
Defi Defini niti tion ons s and and Acr Acron onym yms s Acronym
QA SRD PM PL TL InternationalKids.com
Description
Quality Assurance Software Requirement Document Project Manager Project Lead Technical Lead International-Kids.com
1.17Assumptions and Dependencies • • •
•
•
•
•
Build will be released on time for testing as per the plan. All “Show-Stopper” bugs receive immediate attention from the development team. Testing of all available features in International-Kids.com application will be done using data dump provided by DBA. Enhancements will be incorporated into the original design of the InternationalKids.com application. Some enhancements will require further analysis. This analysis will be worked out during construction phase. All bugs that are prioritized for next version will be unit tested and fixed by the development team before the next version can be released. Functionality of the system will be delivered as per schedule and specifications for the testing team during each phase
Page 7 of 35
INTERNATIONALKIDS.COM •
•
1
Requir Required ed resources resources will be available. available. Project Project Manager will ensure ensure availability availability of environment. Once the PA code enters DBA’s development environment, all bugs will be tracked using Bugzilla which is DBA’s Bug tracking tool. All bugs will be tracked under RIS project in Bugzilla.
Test Re uirement 2.1
1. General requirements
Features to be Tested
•
•
2. Roles 3. Site entry/exit 4.Application Submission
5. Peer Review
6. General Administration 7. Email Email Func Functio tions ns
• •
General requirements for Landing pages of various Roles/Users (International-Kids.com landing page requirements) Dashboard functionality Functionality/ Role of each User in particular External/Internal user (Login/Logout)
Registration (Update/view profile page) Applicant level validation • Update/View/Re-submit • • Acceptance / Rejection of Application by Moderator. Moderator. • Approval of Application by International-Kids.com -Admin Staff Assignment to Committees • Identification of Primary and Secondary Conflicts • Assignment to Reviewers Reviewers • Dual Application • Brokering Brokering list • Reviewer/s • Access to submitted applications • Preferences • Add/update Preliminary Scoring/Triage/Critiques/Concerns Scoring/Triage/Critiques/Concerns • Meeting • Critique Editing by (Reviewers and Admin) • Concern Generation/Resolution • Generation of Committee Scores/Final Scores/Median/Variance Scores/Median/Variance • Average Merit & Percentile Score generation • • NEW/Existing users Maintenance/Management Access control/Roles and Responsibilities(Security Responsibilities(Security • maintenance) ( Initial Initial Notif Notifica ication tions, s, , Maintai Maintain n Message Message Tex Text) t) •
All the features to be tested will be detailed in respective Test Scenario documents and Test Test case documents based bas ed on test types mentioned in Scope section.
Page 8 of 35
INTERNATIONALKIDS.COM All the Test Test Scenario documents will be delivered for review during Pre-construction phase and the Test case documents will be delivered in the middle of Construction phase just before integration test begins. Please refer to “Deliverables” section below for deliverable dates.
Milestones (Schedule) NOTE: NOTE: Fo Follo llowi wing ng dates dates are projec projected ted with with the assump assumptio tion n of begin beginnin ning g the th Construction phase on 10 Oct 2007. Actual dates will be modified as per the Project plan once the construction phase begins. Task Name
Ramp up Understand requirements and review docs. Test Plan Development Test Plan Review/Updation Sign off on Test plan Test Scenario Development FunctionalArea1: FunctionalArea1: Application Submission Review and updation FunctionalArea2: Peer Review Review and updation Non functionality requirements requirements (Performance, Security, e.t.c) Review and updation Test Case Development FunctionalArea1: FunctionalArea1: Application Submission Review and updation FunctionalArea2: Peer Review Review and updation Non functionality requirements requirements (Performance, Security, e.t.c) Review and Updation Establish Testing Environment Smoke Testing Functionality Testing (Includes Functionality, Gui, Usability, Security and Database testing) Regression Testing (Includes Regression, Adhoc and Defect fix verification testing) Integration Testing (Includes External Interfaces and Module Interfaces testing) Access to International-Kids.com QA Environment
Durati on
Start Date
End Date
10-Oct-2006 21-Oct-2006
11-Oct-2007 12-Oct-2007 15-Oct-2007 16-Oct-2007 17-Oct-2007 18-Oct-2007 11-Oct-2007 12-Oct-2007 15-Oct-2007 16-Oct-2007 17-Oct-2007 18-Oct-2007 19-Oct-2007 22-Oct-2007 23/24/25-Oct2007 26-Oct-2007 27/29-Oct2007
Page 9 of 35
INTERNATIONALKIDS.COM System Testing (Includes all types of testing mentioned above and the test types mentioned below) Compatibility Testing Performance and Load/Volume Load/Volume Testing User Acceptance Testing
2
30,31-Oct2007, 1-Nov2007
Testi sting Envi nviro ronm nmen entt
1.18 Browsers Browser
Internet Explorer 6.0 Internet Explorer 7.0 Mozilla Fire Fox Opera
Execution
Environment
√
Win XP
Cert Certiific ficatio ation n
Win XP
Certification Certification
Win XP Win XP
Execution of Test cases:
“√” Symbol mentioned above refers to - “ Entire test cases will be executed ”. The text , “Certification” “Certification” mentioned above refers refers to - “Selected test cases will be executed to verify the capability of the application application on these browsers”.
1.19 Hardware and Software Requirements 3.2.1
Offshore
This section describes the environment setup offshore that is used in the development and testing of the application. 1.1.1. 1.1.1.1 1
Develo Developme pment/ nt/ Deve Develop lopmen mentt Integr Integrati ation on Envir Environm onment ent::
The Development Environment offshore corresponds to the environment used by the developers during the construction. Unit Testing on International-kids.com version is perfor performed med on this this envir environme onment. nt. Each Each develo developer per machin machine e will will have have Internationalkids.com running on Apache Web server. There will be a common MYSQL development database server and all the developers will be using the same database server. server.
Page 10 of 35
INTERNATIONALKIDS.COM SOFTWARE Type
Name
Version
Web Server
Apache
Front End Designing Tool Scripting Language
PHP(Personal Home Page to Hypertext Preprocessor) Java script and Ajax
Database
MYSQL
Browser
IE
Apache2. 0 PHP 4.0/5.0 Java Script Ajax MYSQL 5.0 6.0
OS
Windows XP Windows XP Windows XP Windows 2000 Server Windows 2000 Professional Windows XP
HARDWARE Machine type
HDD
RAM
Web server
40 GB
1 GB
Database Server(MYSQL)
80 GB
1 GB
1.1.1.2
CPU
Intel Pentium 4, 2.66GHz Intel Pentium 4, 2.8 GHz
QA
The QA Environment offshore corresponds to the t he environment on which Integration Testing is performed for International-kids.com version. Offshore QA SOFTWARE Type Web server Scripting Language Database Browser (Base) Browser (Certification)
Name
Version
Apache Java Script, Ajax
Apache 2.0 Java script Ajax MYSQL5.0 6.0 7.0 6.0/7.0 4.0/5.0 9.22
MYSQL IE IE Mozilla Fire Fox Opera
OS
Windows XP Windows XP Windows 2000 Server Windows XP Windows XP Windows XP Windows XP Windows XP
HARDWARE Machine Type
HDD
QA web server
40 GB
RAM
1 GB
CPU
Intel Pentium
OS
Browser
Windows XP
Page 11 of 35
INTERNATIONALKIDS.COM QA Database Server(MYSQL) Test 1 (Desktop class) / Base Test 2 (Desktop class) / Certification Bugzilla Server
280 GB 40 GB
1 GB
80 GB
1 GB
40 GB
1 GB
1 GB
4, 2.8 GHz Intel Pentium 4, 2.8 GHz Intel Pentium 4, 2.4 GHz
Professional Windows 2000 Server Windows XP Professional
Intel Pentium 4, 2.4 GHz Intel Pentium 4, 2.66 GHz
Windows XP Professional Windows XP Professional
IE6.0 IE 7.0
1.20Human Resources
Resource Title
QA lead Test Test Engineer E ngineer (For performing Testing of typesFunctionality, Gui/Usability, Database, Smoke, Sm oke, Regression) Regression) Test Test Engineer E ngineer (For performing Testing of typesPerformance, Load/Volume, Compatibility, Security, Adhoc)
3
Number
Date Required
Resource name
1 2
1
Role Roles s And And Res Respo pons nsib ibil ilit itie ies s
Role
QA lead (Team member)
Responsibility
•
•
• •
•
•
Preparing/Updating the Test plan Preparing/Updating the Test Scenarios Reviewing the Test cases Building/deploying the application in QA/System integration environment Preparation of Traceability matrix Daily Test plan preparation*** (Please refer below for details)
Work Ph, Mobile, Email id
Name
Secondary Role
Project Lead / Project Manager
Page 12 of 35
INTERNATIONALKIDS.COM •
Tester 1/2/3 (Team member)
•
• •
•
•
•
•
•
Generating Test summery report Preparing/updating Test cases Reviewing Reviewing the test cases Executing test cases in Integration / System environment Recording Recording test results in Integration/System environment Impact analysis for failed test cases Logging/verifying/closing and tracking defects Raising issues/clarifications in Issue tracker/clarification register on Bugzilla. Perform various types of testing like Functionality, Smoke, Regression, Adhoc, Security, GUI/Usability, Volume, Compatibility, Performance/Load, Database.
QA lead
Daily Test plan preparation***
The QA lead is responsible for preparing the daily test plan that shall include the following: 1. Alloca Allocate te the the worklo workload ad for each each tester tester 2. Plan to ensure ensure that that the tests being being performed performed will will cover all all require required d functionality for the required OS and Browser. 3. Create Create and maintain maintain a list list that defines defines the the range of script scripts/test s/test cases cases to be completed on specific days 4. Distribute/ Distribute/communi communicate cate this list to the the testers testers 5. Distribute Distribute the the Daily Daily test test plan to to the Projec Projectt Manager Manager
4
Test St Strategy
Page 13 of 35
INTERNATIONALKIDS.COM The Test Test Strategy presents the recommended approach a pproach to the testing of the International-kids.com Development Project. The previous section on Test Requirements described what would be tested; this describes how it will be tested.
Test Process Workflow
The above diagram diagra m explains the complete QA process/Test process/Test life Cycle in General. Following are the steps which would explain in detail , the Test Strategy to be followed for International-kids.com Application. Step1. Test Organize/Review Project Documentation Step2. Test plan Step3. Test Design/Development Step4. Unit Test Execution Step5. Integration/System Test Execution Step6. Defect Tracking and Management Step7. Update Documents and Results Step8. Test Reports Step9. UAT and Closure
Page 14 of 35
INTERNATIONALKIDS.COM 1.21T 1.21 Test Organize/Review Project Documentation Documentation reviews provide a means for testing the accuracy and completeness of the planning, requirements and specifications. Throughout the project, periodic reviews will be held to assure a ssure the quality of project documentation. These reviews will: •
Ensure Ensure projec projectt plans plans have have adequat adequate e time time alloca allocated ted for testin testing g activi activitie ties s and determine limitations.
•
Ensure that the Business Requirements, Requirements, Information Site Flow, Use Cases, Business Rules, and Technical Design documents clearly articulate the functionality of the International-kids.com. -kids.com.
1.22Develop System Test Plan This step of the testing process involves creation of the System Test Test Plan (this document). This will serve as the guidepost for development of test cases and for integration of testing with other project activities. •
•
This plan describes at a high level the overall testing plan and strategy for the International-kids.com Application. Professional Access will follow this plan to develop test scenarios/cases and scripts that will be used for system testing.
•
Test Test scenarios will be described in separate document(s).
•
Test Test Cases will be described in separate document(s)
•
Profe Professi ssional onal Acces Access s will will obtain obtain test test account accounts s and Ids for Interf Interface ace testin testing g (see (see Scope).
1.23T 1.23 Test Design/Development
Page 15 of 35
INTERNATIONALKIDS.COM
Brief explanation of Test Design/Development workflow with respect to the Process flow diagram displayed above: T1,T2 T1,T2
Test lead lead takes takes part in preparat preparation ion of Elaborat Elaboration ion phase phase deliver deliverabl ables es like like Test Test plan and updates the Artifacts in CVS for further reference
T3, T4
From Post Post elaboration elaboration phase to PrePre-constr construction uction phase, phase, Test Scenarios Scenarios would would be designed by the Test lead for the modules/features available in SRD. When once final draft version of SRD with all modules/features specification/s is/are received, Test scenarios are designed and completed during Pre-construction phase. All the created/updated Test Test scenarios are hoarded in CVS T5 Test lead assigns the task of test case/test script creation to test team members during the Construction phase phase T6, T6, T7 For all all the the Test est scen scenar ario ios s crea create ted d earl earlie ierr du duri ring ng Postost-el elab abor orat atio ion/ n/P Precons constr truc ucti tion on ph phas ase, e, the the Test est team team memb member ers s desi design gn test test case cases s du duri ring ng Construction Phase. All the created Test Test cases/Test cases/Test scripts are hoarded in CVS T8, T8, T9 All the create created d Test cases/T cases/Test est Scripts Scripts are reviewe reviewed d by Test Test lead and all the review comments are updated in CVS T10 T10 Test est team team memb member ers s will will che check ck the the rev revie iew w comm commen ents ts and and update update res respe pect ctiv ive e test test cases and hoard the same in CVS. T11 T11 Test est Lead Lead will will map the the requ requir irem emen ents ts to Tes Testt cases cases in Tra Trace ceab abil ilit ity y matrix matrix (The (The objective of this matrix is to illustrate how to document which test case/s test which function functionality ality of software software and which which structural structural attribute. attribute. It maps test requirements requirements to the test cases/T cases/ Test Scenarios that implement them.)
Written test cases and scripts will be used to direct system testing efforts. Professional Access test team will write these in accordance with the System Syst em Test Test Plan. •
Tests Tests will be developed to exercise the required functionality for the website, validate data integrity, and ensure that data is passed or received successfully
Page 16 of 35
INTERNATIONALKIDS.COM from external interfaces. Test Cases will be written in a separate document appended to this plan. •
Each test case will document the steps or actions required to exercise a specified area of functionality. The test cases will be reviewed to verify that they properly validate the intended functionality. functionality. Actual testing will be performed by executing executing the steps of the test case. A pass/fail notation will be made for each step.
•
Each test case will be executed manually and using automated testing tool(for Perfor erforma manc nce/ e/Lo Load ad testi testing ng)) usin using g the the brows browser er vers versio ions ns ment mentio ione ned d in Test est environmen environmentt section. section. A pass/fa pass/fail il notatio notation n will will be recor recorded ded for each each condit condition ion tested, noting the severity and reason for each instance of failure. Test scripts to perfor perform m Perfor Performan mance/ ce/Loa Load d testin testing g will will be execu executed ted automati automatical cally ly during during the System testing phase.
1.24Unit Test Execution Unit Unit test testin ing g veri verifi fies es each each modul module, e, comp compon onen ent, t, obje object ct,, or prog program ram deve develo lope ped d is functionally correct correct and conforms to requirements. requirements. A unit is defined as a single program program function function in terms of inputs, processes processes and outputs. outputs. A program unit unit is small enough that the developer who developed it can test it in great detail. The developer that wrote the code is responsible for creating, updating and executing the unit un it tests tests afte afterr each each succ succes essfu sfull bu buil ild d in the the deve develo lopm pmen entt envi enviro ronm nmen ent. t. Sepa Separat rate e document has been prepared drafting Unit test strategy. strategy.
1.25Integration/System 1.25 Integration/System Test Test Execution 1.25.1 1.25.1 Integra Integratio tion n Test Testing ing
The objective of these tests is to ensure that all the components of the system function properly together and that the application interfaces properly with external applications. Entrance Criteria
All functions to be tested have successfully passed unit t esting All Severity 1 and 2 defects are fixed and have successfully passed unit testing. (See Defect Management portion of this document for severity definitions) Software build is properly version controlled Build report has been completed and submitted with build All hardware and software configurations are in place and ready to test All test cases required for integration testing have been prepared All required integrated systems are available
Exit Criteria
Page 17 of 35
INTERNATIONALKIDS.COM
All components delivered and tested function as detailed in the documents in the References References portion of this document Test Test cases have been updated if and when functionality has changed Test Test results report is developed/updated All new defects have been logged into the issues tracking databa se
1.25.2 1.25.2 System System Testing esting
The test team will conduct a system test to verify that the software matches the defined require requirements. ments. Once the application application has execute executed d successfully successfully under integration integration test, each test suite will be executed against the other supported configurations to ensure defects are not created because the the system configuration has changed. A separate test environment must be established for all hardware, software, and browser configurations supported Entrance Criteria
All functions tested have successfully passed integration testing All severity severity 1 and 2 defects defects are fixed and have successfully successfully passed regression regression testing Test Test cases have been updated if and when functionality has changed All test cases required for system testing t esting have been prepared All hardware and software configurations are in place and ready to test All required integrated systems are available
Exit Criteria
5.6. 5.6.3 3
All Severity 1 and 2 defects are fixed and have successfully passed regression testing The The risks risks associa associated ted with with not corre correcti cting ng any outstan outstandin ding g Severi Severity ty 3and 3and 4 defects have been identified and signed off by the Project Manager, Technical Lead, QA Lead All components delivered and tested function as detailed in the documents in the References References portion of this document Regression Regression tests have been performed and executed successfully Test Test results report is developed/updated All new defects have been logged into the issues tracking databa se
Testi esting ng Types ypes
The scope of the work is to conduct testing in the following areas: ♦ ♦ ♦ ♦ ♦ ♦
Functionality Database Smoke Security User Interface/Usability Compatibility
Page 18 of 35
INTERNATIONALKIDS.COM ♦ ♦ ♦
♦
Performance/Load/Volume Adhoc Regression
Functionality Testing
The objective objective of this test is to ensure that each element of the application application meets the functional requirements of the business as outlined in the: Software Requirement Document/Use cases. Software Design Document. Other Functional documents produced during the course of the project i.e. resolution to issues/change requests/clarifications/feedback. Secondly, includes includes specific functional testing, which aims to test individual process process and data flows. This stage will also include include Validation alidation Testing, esting, which which is intensive testing of the new front-end fields and screens. Functionality testing will be performed on every build that is right from when Build series (2 weeks Test process cycle) commences till the final System-testing pass. In other words, Functionality testing will be performed by testing team just after the development of set of features as per the decision of Technical lead/Project manager, basic basical ally ly as part part of inte integr grati ation on testi testing ng.. This This proc proces ess s will will be conti continu nued ed till till the the completion of System testing phase. ♦
Database Testing
Database would be tested under following f ollowing perspectives:
Test Testing ing the database database Sch Schema ema (Store (Stored d proce procedur dures, es, trigge triggers, rs, views views e.t.c) e.t.c) after after migration (done by MYSQL DBA developer) Test Testing ing the databas database e which which houses houses the conten contentt that that the International-kids.com application manages, runs queries and fulfills user requests for data storage (done by Testing team) Issues to test are: Data integrity errors (missing or wrong data in tables) Outp Output ut erro errors rs (err (error ors s in writ writin ing, g, edit editin ing g or readi reading ng/r /ret etri riev evin ing/ g/qu quer eryi ying ng operations in the tables)
Database testing will be performed along with functionality testing on every build, right from the First Build series (2 weeks test process cycle) till the final Build series. ♦
User Interface/Usability Testing
The usability testing will be accomplished by verifying the information in each window is accurate. Menus, icons and toolbar functionality functionality will be tested tested as applicable to the navigation and results panes. Importance will be given to graphics, contents, data presentation, feedback and error messages, design approach, user interface controls, formatt formatting ing,, instru instructi ctions ons e.t.c. e.t.c. Multi Multi Windo Window w Overla Overlappi pping ng will will be tested tested becaus because e product supports opening of multiple documents. GUI/Usability testing will be performed along with functionality testing on every build, right from the First Build series (2 weeks test process cycle) till the final Build series.
Page 19 of 35
INTERNATIONALKIDS.COM ♦
Adhoc Testing
Adhoc Testing is done on every build right from first Build series till the last Build series. This is mostly experience based testing and carried out from the application usage perspective. Just based on knowledge of functionality/ies the test team member will will perfor perform m this this test. test. He/she He/she need need not refer refer to any Test case/S case/Scen cenari ario/Pl o/Plan. an. User User concentrates on navigations that are unusual, negative or across the components. During During the second second week week of every every Build Build series series (2 week week test test proce process ss cyc cycle) le) Adhoc testing will be performed. This test will be performed during Integration test phase as well as System test phase. ♦
Smoke Testing
During Integration testing, which is performed in parallel with development phase, every time before releasing the Build to QA team, Development team performs Smoke testing to check whether mentioned/planned set of features have been implemented without getting into details. When once the build is released to QA team, before accepting the Build for further testing process, Smoke testing is performed to check whether the application’s planned set of most crucial functionalities work, without bothering with finer details. It does mean that for a released build, availability of all the features as mentioned in Release notes will be tested. When once the System passes smoke test, it would be subjected for further tests. Before commencing System testing too, QA team would perform smoke testing to check whether all functionalities are implemented into the System at high level. ♦
Compatibility testing
Browsers: Compatibility matrix where different brands and versions of browsers are tested to a certain number of components and settings, settin gs, for example Applets, client side scripting, ActiveX controls, HTML specifications, Graphics or B rowser settings, has/have been mentioned in Section 3.2 • Settings, Preferences: Preferences: Depending on settings and preferences of the client machine, web application may behave differently. Options such as screen resolution and color depth would be considered while testing. • Printing: Despite the paperless society the web was to introduce, printing is done more than ever. Testing would be performed to check whether the pages are printable with considerations on: Test and image alignment Colors of text, foreground and background Scalability to fit paper size, e.t.c Selected set of Usability/GUI test cases will be executed as a part of Compatibility testing during System testing phase. •
♦
Security testing
Security testis will determine how secure the new AHA-RSDP system is. The tests will verify that unauthorized user access to confidential data is prevented. This type of testing would be performed to check
Page 20 of 35
INTERNATIONALKIDS.COM
♦
That for each known user type the appropriate function / data are available and all transacti transactions ons functi function on as expec expected ted and run in prior prior Applic Applicatio ation n Function Function tests t ests Directory setup That without authorization, access permissions will not be provided to edit scripts on the server Time-out limit Bypassing login page by typing URL to a page inside directly in the browser, e.t.c
Performance and Load/Volume Load/Volume testing Performanc Performance e testing testing verifies verifies the application application’s ’s response response time under maximum load conditions. conditions. The purpose purpose of performance performance testing is to measure the applicatio application n under load condition conditions. s. Subjecting Subjecting the the applicati application on system system to expected expected peak loads loads before before release can ensure software software quality. quality. Questions answered are:
(1) Do applications and databases perform correctly correctly under load? (2) What response time can be expected expected and will it meet meet requirements? (3) What operations negatively impact impact performance? Performance testing procedures
The general approach approach for load testing is to set up a test website configuratio configuration n and to run selected selected test scripts against against it to measure measure performance. performance. The configuration configuration and test environment environment should should mirror the production production environmen environment. t. Individual Individual tests will be run to verify correct operation of the scripts. Then the scripts will be run again in severa severall cyc cycles les.. Each Each cyc cycle le will will incre increase ase the num number ber of concur concurre rent nt users users until until the required system capacity has been successfully demonstrated. The The test testin ing g proc proces ess s is inhe inherrentl ently y iter iterat ativ ive; e; sinc since e earl early y test tests s may may enco encoun unte terr bottlenecks or defects. The tests will need to be repeated after the system has been tuned tuned or reco reconf nfig igur ured ed or the the defe defect cts s have have been been corr correc ecte ted. d. In many many case cases, s, one one bottleneck may obscure the presence of another. Thus, when problems have once been corrected, it is possible (even likely) to encounter others on subsequent trials.
The goal of performance testing is to be able to: 1) Determine Determine if the customer customer will experi experience ence unacceptab unacceptable le response response time when the store website is under load. 2) Determine Determine if the the web server server,, application application server server or database database server will crash under load. 3) Tune the applica application tion based on performa performance nce issues issues found. Metrics that we will attempt to achieve include:
The response time from the point when the web server receives a page request to the point when the web server serves the requested page is a metric used to test performance. This metric will be revisited revisited once the pages have been built to determine an acceptable response time. There will be separate metrics for the search results pages vs. the other pages. concurrent users *
Page 21 of 35
INTERNATIONALKIDS.COM active users** * Concurrent users refers to users who are maintaining an active session with the site and and may may or may may not not be acti active vely ly clic clicki king ng on the the site site.. (Ple (Pleas ase e see see Techn echnic ical al Specification for details) ** Active users refers to those users who are actually clicking on the site at any given time. (Please see Technical Technical Specification for details)
Areas of the website that we recommend load testing include Site Area Concurrent Action Tested Home Page - Where multiple users access the home page Registration - Where multiple users Register using different usernames - Where multiple registered users login Search/Brow - Where multiple se users Browse same category - Where multiple users Browse different categories - Where multiple users Search various keywords General site - Where multiple Navigation users navigate general site functionality
♦
:
Load Area Page loading/performance Updating the database Accessing the database Accessing the database Accessing the database Accessing the database Page loading/performance
Regression testing
A Regression test will be performed subsequent to the release of each Build from second release on wards to ensure that
5.6. 5.6.4 4
There is no impact on previously released software with the addition of new functionality, and To ensure that there is an increase in the functionality and stability of the software. There is no impact on previously released software with the resolution of defects.
Test est Exec Execut utio ion n wo workf rkflo low w
Page 22 of 35
INTERNATIONALKIDS.COM Test Method: M ethod:
The following activities will be performed during the test process: The The develo developme pment nt team will will verify verify throug through h unit unit testin testing g that that each each module, module, component, object, and program is functionally correct and conforms to the use case definitions document.
The test team will conduct conduct a functionali functionality/inte ty/integration gration test of the larger system to ensu ensure re that that all all the the func functi tion onali aliti ties es/c /comp ompon onen ents ts of the the syst system em func functi tion on properly properly together and that the application application interfaces interfaces properly properly with external external application/s.
The test team will conduct a system test to verify that the software matches the defined requirements. All the test cases/scripts executed during previous QA cycles will be re-executed to check the correctness of the system. Once the application has executed successfully under integration test, each test suite will be executed against the other supported configurations to ensure defects are not created created because because the system configuration configuration has changed. changed. A separate test test envir environm onment ent must must be establ establish ished ed for all hardwa hardware re,, softwar software, e, and brows browser er configurations supported.
The The test test team team recom ecomme mend nds s that that perf perfor orma manc nce e test testin ing g be done done usin using g perfor performanc mance e tool. The purpose purpose of load testing testing is to ensure ensure stability stability of the application application under under simulated load conditions. conditions. Automated Automated performance performance testing testing tool tools s can can simul simulate ate the the load load on the the syste system m bein being g test tested ed,, elim elimin inat atin ing g the the nece necessi ssity ty of empl employ oyin ing g hu hund ndrreds eds of user users, s, hu huge ge volu volume me of data, data, many many transactions and obtaining the required required equipment.
The test team will conduct the tests by executing the test cases and scripts. Each Each test test case case will will test test a specif specific ic area of functio functionali nality ty.. Test cases will be comprised comprised of several test scripts that detail that functional functionality ity.. The test cases will be reviewed to ensure that they cover the scenarios needed to adequately test the site and its functionality. functionality.
Each Each test case will have have an expec expected ted resul resultt and a pass/fail pass/fail column. column. If the expected result is achieved a value of “Y” will be recorded in the actual results column. If the expected result is not achieved achieved a value of “N” will be recorded in in the actual results column, and the defect will be logged in the issue-tracking database. The actions that led to the failure and an assessment of its its severity will also be noted in the issue-tracking database.
The development development team will fix defects based on the level level of severity assigned by the test team. The defect information will will be recorded recorded in the issue-tracking database (Bugzilla), and the developers will be informed of each new issue via email. email. The severi severity ty levels levels to be used during during the test are descri described bed in the Defect Management portion of this document.
The test team will receive notification via email after each defect has been corrected and unit tested by the development development team. The test team will retest retest the defect by re-executing the test case and script in which the defect was found. The regression regression test will verify verify that the altered altered code has not adversely impacted previously working functionality. functionality.
Page 23 of 35
INTERNATIONALKIDS.COM
The test team will track all the test cases and test scripts using a Traceability document.
Included within the scope of the test is an external interface test, designed to verif verify y that all compon component ents s provi provided ded by third third party party provi provider ders s inter interfac face e and interact according to specifications.
A Separate test environment will be established for all hardware, software and brow browse serr conf confiigu gura rati tion ons s supp suppor orte ted. d. Refe efer to Har Hardwar dware e and and sof softwar tware e requirements requirements section for more information.
Following diagram explains the flow of test types/phases followed for Internationalkids.com Application.
Page 24 of 35
Page 25 of 35
INTERNATIONAL-KIDS.COM Test Flow:
Testing of International-kids.com application would be performed at feature level. A two week internal build release approach will be adopted while testing. Inte Integr grat atio ion/ n/F Fun unct ctio iona nali lity ty Testi esting ng star starts ts as soon soon as the the firs firstt set set of feat featur ures es is developed/released by the development team by adopting build series procedure. This process continues till the completion of System Testing. Development team will decide and inform testing team about the set of feature that are planned for every build release so that Test Scenarios/Test Scenarios/Test cases would be developed and a nd reviewed well in advance The typical flow of activities that happen in a 2-week QA test process cycle (Build Series) can be summarized through the Table given below. Day Series Phase Activities Monday Start of Build Build Series N: Test initialization activities Series N Build Series N: Receive Build N and Release notes by 1 P.M Build Series N: Deploy the build Build Series N: Run Smoke test cases and Round 1 testing Tuesday
INTERNATIONAL-KIDS.COM Test Flow:
Testing of International-kids.com application would be performed at feature level. A two week internal build release approach will be adopted while testing. Inte Integr grat atio ion/ n/F Fun unct ctio iona nali lity ty Testi esting ng star starts ts as soon soon as the the firs firstt set set of feat featur ures es is developed/released by the development team by adopting build series procedure. This process continues till the completion of System Testing. Development team will decide and inform testing team about the set of feature that are planned for every build release so that Test Scenarios/Test Scenarios/Test cases would be developed and a nd reviewed well in advance The typical flow of activities that happen in a 2-week QA test process cycle (Build Series) can be summarized through the Table given below. Day Series Phase Activities Monday Start of Build Build Series N: Test initialization activities Series N Build Series N: Receive Build N and Release notes by 1 P.M Build Series N: Deploy the build Build Series N: Run Smoke test cases and Round 1 testing Tuesday Build Series N: Round 1 Testing Wednesda Build Series N: Round 1 Testing y Thursday Build Series N: Round 1 Testing Friday Build Series N: End Round 1 Testing Build Series N+1: Features/Mo Features/Modules dules acquisition, planning, effort estimation, resource allocation Saturday Sunday Monday Build Series N: Start Round 2 Testing Tuesday Build Series N: Round 2 Testing Wednesda Build Series N: Round 2 Testing y Build Series N+1: Submit Test Scenarios/Cases for Review Thursday Build Series N: Round 2 Testing Friday End of Build Build series N: Test summery/conclusion Series N report generation by the end of day Build Series N+1: Update test cases based on review feedback, prepare for series N+1 Assuming that the t he Test Test case/scripts Execution process begins from 15- Jan-2007(subjected to change) , QA team will execute the following testing cycles by considering 1) Smok Smoke Tes Testt pass pass 2) Functionality unctionality test pass(which pass(which includes includes testing types like like Functional Functionality, ity, GUI/Usability, Database and non-functionality test type like Security) These are in turn divided into two categories based on builds “1”- Execution of test cases of features included in current build “2”- Re-Execution Re-Execution of test cases of features included in all al l the previous builds 3) Integration Integration test pass(which pass(which include includes s testing types types like module module interface interfaces s and External interfaces)
Page 26 of 35
INTERNATIONAL-KIDS.COM 4) Regres Regression sion test pass(which pass(which includes includes testing types types like Regre Regression, ssion, Adhoc Adhoc and defect fix verification) 5) System test test pass( which which includes includes all the above above types of testing testing and testing testing types types like Performance, Load/Volume, Load/Volume, Compatibility Compat ibility ) QA Cycle / Build Series 1 2 3
4 5 6 7
8 9 10 11
Durati on
2 weeks 2 weeks 2 weeks 1 week Buffer time 2 weeks 2 weeks 2 weeks 2 weeks 1 week Buffer time 2 weeks 2 weeks 2 weeks 2 weeks
Smok Functionali e ty 1 2
Integratio n 1 2
Regr essio n
Syste m
Test Cases/ Scenarios Updation/ Addition
1.26Defect Tracking and Management The defect management process ensures maximum efficiency for defect recognition and resolution. The objectives of this process are:
To To maintain a defect tracking system to reliably monitor defects and fixes. To preserve a history of defects and their fixes. To To ensure prompt and efficient identification and notification of defects. To provide timely fixes and deployment.
QA team will use Bugzilla (a defect tracking tool), which will allow PA developers and QA members to carry out a full test cycle: find, f ind, log, assign, fix, verify, resolve, and close. The number of defects that surface during the QA testing period, including their potential impacts and complexity to implement, can be quite unpredictable. The PA Technical Lead / Project Manager will respond to defects in the minimum time possible, and assign fixes to a particular build. Careful review of the impact of an implemented fix will minimize reoccurrence and/or t he introduction of new problems. However, since testing alone cannot fully verify that software is complete and correct, PA takes a comprehensive validation approach. QA processes are integrated into all stages of the PA Development from the start of the engagement (e.g., large scale planning, unit testing, etc.).
Page 27 of 35
INTERNATIONAL-KIDS.COM Bugzilla defect tracking tool will be used for defect tracking and reporting purpose. It can be accessed via the web: URL = • Project name = International-kids.com • Each team member will be given a User ID and Password • Following are activities performed during Defect tracking process: 1) A Test Test engineer engineer executes executes the test case/script case/script and compares compares the actual result result with the expected. expected. He/She He/She enters test results results under under results results column in test case document document across each test case by marking ma rking “Pass”/”Fail”. “Pass”/”Fail”. 1. When When a test case fails fails,, after after resul resultt is updated updated in test test case docume document, nt, A defect defect is entered into Bugzilla and the corresponding defect reference number is mentioned in the test report (test case document used for testing). 2) Following Following informati information on is entered entered for every defect defect in each defect defect report: report: 1. Bug nu numb mbe er 2. Summery 3. Desc Descri ript ptiion 4. Steps Steps to re-c re-cre reate ate the the pro proble blem m 5. Attac Attachme hments nts if any any 6. Configuration Configuration the problem problem was was found found in (Browser/ (Browser/Os/ve Os/version) rsion) 7. Function/comp unction/component/mod onent/module ule the the problem problem was found found in 8. Seve Severi rity ty of of pro probl blem em 9. Owne Owner/ r/As Assi signe gned d to 10.URL 11.Status 12.Submit 12. Submit Date Date 13.Submitter/Reporter 14.Resolution 2. The defect defect is assign assigned ed to the QA lead, lead, who will in turn turn monitor monitor all the defec defects ts for completeness before submission to Development Tech Lead. 3. All defects defects will be check checked ed for dup duplic licate ate defects defects in Bugzilla Bugzilla before before submissi submission on to Development Tech Lead. 4. Defects Defects should be reproduci reproducible ble before before being submitted submitted to development development Tech Tech lead. 5. QA lead will will monitor all all defects defects that are in the escalati escalation on process. process. The defects defects will be classified, managed and escalated using a process agreed upon between AHA and Professional Access. 6. Tech lead along with module module lead will review review the defects. If a defect is valid defect, Tech lead will assign it to respective developer or else reject it by specifying the reason and re-assign it to respective reporter/submitter 7. Defect Defects s will be fixe fixed d based on sever severity ity.. Those Those defect defects s entere entered d as a Severity Severity 1 (Cri (Criti tica cal/ l/Sh Show owsto stoppe pper) r),, or Seve Severi rity ty 2 (Hig (High) h) must must be corr correc ecte ted d prio priorr to the the application application being deployed. deployed. Severity Severity 3 defects defects (Medium) will be corrected corrected based on consensus agreement between Project Manager, Technical Lead and QA Test Lead regarding their criticality. 8. The person person,, who has been been assigned assigned the defect, defect, carries carries out the impact impact analysis analysis (identifies the cause of the problem, identifies the impacted components and also identifies the fix to be carried out) and then fixes the defect appropriately. He records the impact analysis briefly in the Bugzilla.
Page 28 of 35
INTERNATIONAL-KIDS.COM 9. Integr Integrati ation/S on/Syst ystem em test cases cases are updated updated if the defect defect has been escaped escaped due to the lack of correspond corresponding ing Integration Integration/Syste /System m test case and Integration/S Integration/System ystem testing that was carried out by respective Submitter/Reporter. Submitter/Reporter. 10. Defects if any are captured and tracked for closure using the Bugzilla. 11.The Regression testing is performed by ideally re-running Integration/System tests of the the chang changed ed progr program ams. s. The The modif modifie ied d compo compone nents nts are re-b re-base aseli line ned d on successful conclusion of these tests. 12.The product is re-integrated, revised components are built and re-running of full system and integration testing is carried out. Test cases are re-executed under following circumstances After a fix / a change / an enhancement. • Re-verify Re-verify all functions of each build of application. • No new problem introduced by fix / change ("ripple effect"). • During System Testing. • The diagram below provides an overview of the defect tracking process:
Page 29 of 35
Page 30 of 35
Internation-kids.com Defect Classification: Classification:
Defect Defects s identi identifie fied d by the PA testin testing g team team will will be classi classifie fied d based based on the guidel guideline ines s explained explained in the subsequent sections. sections. Apart from the guideline guidelines, s, the context context of a defect defect also has to be considered for proper classification of the defect. The defects can fall into one of the following categories: Severi ty Level 1
Title
Critical/ Showstopper
Description
Causes global data corruption Missing functionality critical to site operation that was defined in specifications Critical function not operational (typically crash, severe application deficiency or malfunction); no work around exists A defect that would adversely adversely impact the reputation reputation of the client and is a critical business issue
Internation-kids.com Defect Classification: Classification:
Defect Defects s identi identifie fied d by the PA testin testing g team team will will be classi classifie fied d based based on the guidel guideline ines s explained explained in the subsequent sections. sections. Apart from the guideline guidelines, s, the context context of a defect defect also has to be considered for proper classification of the defect. The defects can fall into one of the following categories: Severi ty Level 1
2
Title
Description
Critical/ Showstopper
High/Major
3
Medium/Normal
4
Low/Cosmetic/Min or
Priority:
Causes global data corruption Missing functionality critical to site operation that was defined in specifications Critical function not operational (typically crash, severe application deficiency or malfunction); no work around exists A defect that would adversely adversely impact the reputation reputation of the client and is a critical business issue Component system hang or local data corruption. An Emergency defect that has been determined to have a work around. Non-critical function function not operational; no work around. around. If a work around is determined to be available, then defect will be reclassified as Medium. Slow performance Exampl Examples: es: link link brok broken en but access accessibl ible e via anothe anotherr click click stre stream, am, garb garble led d text text in a para paragr graph aph,, inva invali lid d data data in a fields, unapproved content, and painfully slow downloads. Non-cr Non-criti itical cal functi function on not operati operationa onal; l; workaro workaround und is available Nonessential feature or function is missing or broken Operation is not user friendly or somewhat inconvenient Displ Display ay typos typos or misal misalig ignm nmen ents ts that that do not not affe affect ct system operation Bewildering dialog boxes or instructions Inaccurate spelling or grammar Functional Functional or usage defect, which does not hamper the major usage of the application Disp Displa lay y typo typos s or misal misalig ignm nmen ents ts that that do not not affe affect ct system operation. An incorrect color on an element An incorrect object label Operation is somewhat inconvenient. This This defect defect is not relat related ed to applica applicatio tion n functio functionali nality ty and mainly consists of aesthetic and usage usag e issues
INTERNATIONAL-KIDS.COM Priority describes the importance and order in which a bug should be fixed. The available priorities are: Priority level P1
Priority
High
P2
Medium
P3 P4
Low Very low
Definition
Resolve the defect with immediate effect in very next release: Prevents further testing • • Full feature unavailable • Client request • Severe impact on client Effects other features • Resolve the defect at the earliest, before intermediate release (if any) Normal defect, Resolve before Final Client release Could be fixed based on Triage Enhancements • Necessity of the bug fix for the final Client release •
1.27Update Documents and Results
Update the test scenarios, test cases and scripts if and when functionality workflow changes. Update the test case documents with results (Pass/Fail) , every time when test cases are executed Update the test case documents when there is no test case corresponding to the defect raised due to un usual flows if any. any. Update Traceability Traceability matrix every time when Scenarios/cases are updated or added. Develop the Test Results Report (Daily). Prepare and Review Conclusion Report.
1.28Test Reports Status Reporting
1) Bugzilla Bugzilla will be be used to log bugs. bugs. Bug report report should should have sufficient sufficient informat information ion to reproduce the bug 2) QA testing will will be reported reported to the Project Project manager manager on a daily/weekly daily/weekly by Produci Producing ng Testing Results reports. Test Test Results Reports should include, but is not restricted to the following: Report name Fields to include Individual project Name of tester status report Types of testing performed Number of test cases/scripts executed by him/her Number of test cases/scripts not executed executed by him/her Number of defects logged (valid, Invalid, duplicate)
Page 32 of 35
INTERNATIONAL-KIDS.COM Test case/script execution report
Defe Defec ct sta statu tus s rep repor ortt
Defects requiring escalation report
Number of features available for testing Total Total number of test cases/scripts generated Number of test cases/scripts executed per tester Types of testing performed Percentage Percentage of total tot al test scripts completed Total otal num umbe berr of of def defec ects ts log logged ged Total number of defects verified/closed Total number of open defects Issues if any Total number of Severity 1 Defects Total number of Severity 2 Defects Total number of Severity 3 Defects Components/functional areas affected Date detected Current Status
3) QA team and Project Manager will conduct daily/weekly bug scrub meeting. The
following information will be discussed. Current status vs. planned (are we on schedule?). • Test Test cases/scripts execution execution completed (can be at feature level). • Number of defects open and their severity (Bugzilla). • Summery of QA progress • Issues that need clarification/action •
Conclusion Report
Upon conclusion of the QA test cycle, the QA/Test Lead will document the results of the test phase of the International-kids.com system in in the Conclusi Conclusion on Report. Report. This report report contains information such as:
QA Test Test cycle number, duration and dates List of test cases executed Test team members Test case results Number of defects logged with status Metrics to quantify success of the project Copy of defect log
Test Summary Report
Test Test Summary Report would be a combination of all the above reports to present the final testing status during Intermediate/final Release.
1.29UAT and Closure
International-kids.com to perform User Acceptance Testing for the migrated Internationalappliicati cation on wil will use the the International-kids.com -UA -UAT envi enviro ronm nmen ent. t. The The kids.com appl will make make this this envir environme onment nt availa available ble either either throug through h their their International-kids.com team will
Page 33 of 35
INTERNATIONAL-KIDS.COM hosting provider or will host it internally. The purpose of these tests is to confirm that the system system is develo developed ped accord according ing to the specif specified ied user requi require remen ments ts and is ready ready for operat operation ional al use. use. The followi following ng are are the anticip anticipated ated tasks tasks in making making this this envir environm onment ent available: Apache • PHP,Ajax, Java Script • MYSQL instance with data ready for testing • PA will coordinate with International-kids.com Deployment Specialist for the configuration on the environment and will provide International-kids.com code, consolidated migration scripts that are ready for installation into User Acceptance Environment and will perform resolution of defects from the User Acceptance Testing. Testing. Testing Testing will be deemed complete upon upo n the execution of all of the following: •
•
• •
5
All Functionality/Integration/System test cases. All outstanding issues are reviewed and accepted by the International-kids.com and PA Project Teams. Teams. These issues will include all major, critical, and blocker defects. Received Received signed-off creative and production components. The product is acceptable, which is based upon the following principles: System testing is 100% complete and all fixed issues have been regressed regressed o and closed. Calculation includes the total number of valid open bugs divided by the total o number numb er of bug bugs s in the system. system. Total Total Open Bugs Bugs includ include e bug bugs s that are Unconfirmed, Assigned, New or Reopened, whereas Total Bugs include all bugs in the database with no exclusions. All Major, Critical, and Blocker/showstopper bugs are closed. o 95% must be maintained upon the release of the project. o
Conf Config igur urat atio ion n mana manage geme ment nt
Please refer to the Configuration Management document that explains about the complete Configuration management workflow to be followed.
6
Deliverables
NOTE: NOTE: Fo Follo llowi wing ng dates dates are projec projected ted with with the assump assumptio tion n of begin beginnin ning g the Construction phase on 11th Dec 2006. Actual dates will be modified as per the Project plan once the construction phase begins. Deliverable name Test plan Test Scenarios FunctionalArea1: FunctionalArea1: Application Submission FunctionalArea2: Peer Review FunctionalArea3: Pre Awards FunctionalArea4: Post Awards FunctionalArea5: FunctionalArea5: Reports and Admin Non functionality requirements requirements (Performance, Security, e.t.c) Test Test cases FunctionalArea1: FunctionalArea1: Application Submission FunctionalArea2: Peer Review
Deliverable Date 10- Oct-2007
Page 34 of 35
INTERNATIONAL-KIDS.COM FunctionalArea3: Pre Awards FunctionalArea4: Post Awards FunctionalArea5: FunctionalArea5: Reports and Admin Non functionality requirements requirements (Performance, Security, e.t.c) Test case execution report Test results reports Test Test summary/conclusion report
Page 35 of 35
