Chapter 11: ChIan ptteerg1r6a :t Tieo stnin-ga(n 2/d 2) System Testing Object-Oriented SoftwareConstruction Armin B. Cremers, Sascha Sascha Alda & Tobias Rho (based (based on Brue Bruegge gge & Dutoit Dutoit))
Roadmap for lecture
♦
♦ ♦ ♦
4.7.2006 (Lecture (Lecture “Testing” “Testing” and “Agile Software Software Development”) 6.7.2006 (Practice Talk “T-Mobile”) 11.7.2006 (Exercise, on demand) 18.7.2006 (Written Exam), 18:00
Roadmap for lecture
♦
♦ ♦ ♦
4.7.2006 (Lecture (Lecture “Testing” “Testing” and “Agile Software Software Development”) 6.7.2006 (Practice Talk “T-Mobile”) 11.7.2006 (Exercise, on demand) 18.7.2006 (Written Exam), 18:00
Software Lifecycle Activities ...and their models
Requirements Elicitation
Analysis
Expressed in Terms Terms o f
System Design
Object Design
Implementation
Test Testiing
Structured by
Realized by
Implemented by
Verified by
class... class... class...
Use Case
Ap A p p l i c at atii o n Domain
Sub-
Solution Domain
? class.... ?
Source
Test
Test Team Organizational issues Professional Tester Programmer
Analyst
Test User
Team
Configuration Management
System Designer
too familiar with code
Types of Testing ♦
Unit Testing (last lecture): - Individual subsystem - Carried out by developers (of components) - Goal: Confirm that subsystems is correctly coded and carries out the intended functionality
♦
Integration Testing (mainly this lecture): - Groups of subsystems (collection of classes) and eventually the entire system - Carried out by developers - Goal: Test the interface and the interplay among the subsystem
Types of Testing ♦
System Testing:
The entire system Carried out by developers (testers!) Goal: Determine if the system meets the requirements (functional and global) Functional Testing: Test of functional requirements Performance Testing: Test of non-functional requirements
Acceptance
and Installation Testing:
Evaluates the system delivered by developers Carried out by the client. Goal: Demonstrate that the system meets customer requirements and is ready to use
Integration Testing Strategy Test (sub-)systems (cluster) for problems that arise from subsystem interactions ♦ Assumption: ♦
♦
The entire system is viewed as a collection of subsystems determined during the system and object design. System Decomposition is hierarchical
The order in which the subsystems are selected for testing and integration determines the testing strategy Big bang integration (Nonincremental) Bottom up integration Top down integration Sandwich testing Variations of the above
♦
For the selection use the system decomposition from the System Design
Example: Three Layer Call Hierarchy User Interface
Billing System
Database
Event Service
Network Access
Layer III
Learning
Layer II
Neural Network
Layer I
Integration Testing: Big-Bang Approach Unit Test A
Don’t try this!
Unit Test B Unit Test C
System Test
Unit Test D
All components (units) are first tested individually and then together as a single and entire system: ♦ Pros: ♦
♦
No test stubs (mocks) and drivers are needed
Cons:
Difficult to pinpoint the specific component responsible for the failure
Bottom-up Testing Strategy ♦
The subsystem in the lowest layer of the call hierarchy are tested individually
♦
Then the next subsystems are integrated and tested from the next layer up that call the previously tested subsystems
♦
♦
Increment one subsystem at a time Order of integration depends on importance of subsystem etc.
This is done repeatedly until all subsystems are included in the testing
♦
Infrastructure is tested first
Regression Tests: Rerun previous tests
Only Test Drivers are used to simulate the components of higher layers No Test Stubs!
Bottom-up Integration A Layer III
Test E
C
B
Test B, E, F
E
F
D
G
Test F Test C
Test D,G Test G
Test A, B, C, D, E, F, G
Layer I
Layer I
Pros and Cons of bottom up integration testing ♦
Pros:
♦
Interface faults can be more easily found (the usage of test drivers accomplishes a clear intention of the underlying interfaces of the lower layer) No Stubs are necessary
Cons:
Components of the User Interface are tested last Test cases often hard to derive Faults found in the top layer may lead to changes in the subsystems of lower layers, invalidating previous tests.
Top-down Testing Strategy ♦
Test the top layer of the controlling subsystem first
♦
Then combine all the subsystems that are called by the tested subsystems and test the resulting collection of subsystems
♦ ♦
♦
The skeleton of the program is tested
Increment one subsystem at a time
Do this until all subsystems are incorporated into the test Test Stubs are used to simulate the components of lower layers that have not yet been integrated. No drivers are needed
Top-down Integration Testing A Layer III
C
B
E
Test A
Test A, B, C, D
D
G
F
Test A, B, C, D, E, F, G
Layer III Layer III + II All Layers
Layer I
Layer I
Pros and Cons of top-down integration testing ♦
Pros:
♦
Test cases can be defined in terms of the functionality of the system (functional requirements) More effective for finding faults that are visible to the user
Cons:
Writing stubs can be difficult: Stubs must allow all possible conditions to be tested. Possibly a very large number of stubs may be required
Sandwich Testing Strategy ♦
♦
Combines top-down strategy with bottom-up strategy (parallel testing is possible) The system is view as having three layers A
target layer in the middle A layer above the target (top layer) A layer below the target (bottom layer) Testing converges towards the target layer ♦
No Test Stubs and Drivers are necessary for bottom and top layer
Sandwich Testing Strategy A Layer III
C
B
D
Layer I
Test E E
Bottom Layer Tests
G
Layer I
Test B, E, F Test F
Test D,G Test G
Top Layer
F
Test A,B,C, D
Test A, B, C, D, E, F, G
Pros and Cons of Sandwich Testing ♦
Pros:
♦
Cons:
♦
Top and Bottom Layer Tests can be done in parallel No Stubs and Drivers (saves development time) System implementation needs to be finished Does not test the individual subsystems on the target layer thoroughly before integration (C in the example)
Solution: Modified sandwich testing strategy
Modified Sandwich Testing Strategy ♦
♦
Tests the three layers individually before combining them in incremental tests with one another The individual layer tests consists of three tests:
♦
Target layer test with drivers and stubs Top layer test with stubs Bottom layer test with drivers
The combined Layer Tests consist of two tests:
Top layer accessing target layer (top layer replaces drivers) Bottom accessed by target layer (bottom layer replaces stubs)
Modified Sandwich Testing Strategy A Layer III
Test A
Test A,B Test A,C
Test B
Test A,D
Test C
E
D
C
B
G
F
Layer II
Layer I
Test A, B, C, D
Test D
Test E Test F Test G
Test D,G
Test B, E, F
Test A, B, C, D, E, F, G
Using the Bridge Pattern to enable early Integration Testing ♦ ♦
Usage of Design Patterns supports testing Use the bridge pattern to provide multiple implementations under the same interface.
Client
DBImplementation
DBInterface
Stub Code
Oracle
MySQL
Summary Testing Testing isis still still aa black black art, art, but but many many rules rules and and heuristics heuristics are are available available ♦ Testing consists of component-testing (unit testing, ♦ Testing consists of component-testing (unit testing, integration integration testing) testing) and and system system testing testing ♦ Design Patterns can be used for integration testing ♦ Design Patterns can be used for integration testing ♦ Testing has its own lifecycle ♦ Testing has its own lifecycle ♦ ♦
Excurse: Agile Development Agile Development ♦ Extreme Programming (XP) ♦
Agile Software Development: Different Methods and Approaches
♦
Overall problem in software production:
♦
Rapidly changing requirements Complexity (software, models, notations)
Dissatisfaction with the overheads involved in design methods led to the creation of agile methods. Core issues: Focus on the code rather than the design; Are based on an iterative approach to software development; Are intended to deliver working software quickly and evolve this quickly to meet changing requirements.
Agile Software Development: The Agile Software Development Manifesto ♦
Permanent Customer Involvement
♦
Incremental delivery
♦
Skills of team (members) should be appreciated Should be left to work with their own methods, tools etc.
Embrace change
♦
Software is developed in increments
People not process
♦
Role: Provide and prioriterize new system requirements
Expect system requirements to change, so design the system to accommodate these
Maintain simplicity
Focus on simplicity in both software and development process Work together in eliminate complexity
http://agilemanifesto.org/
Problems with agile methods
z
z
It can be difficult to keep the interest of customers who are involved in the process. Team members may be unsuited to the intense involvement that characterizes agile methods. -
In particular shy and reserved people (“nerds”?)
z
Prioritizing requirements can be difficult when there are multiple stakeholders.
z
Maintaining simplicity requires extra work.
Extreme Programming (Beck, 2000) ♦
♦
Perhaps the best-known and most widely used agile method. Extreme Programming (XP) takes an ‘extreme’ approach to iterative development. New versions may be built several times per day; Increments are delivered to customers every 2 weeks; All tests must be run for every build and the build is only accepted if tests run successfully.
♦ ♦
♦
Programming in pairs Continuously re-prioritizing of requirements (client, customer, users) Client is part of development team and in charge
Life Cycle of XP
♦
♦
♦
In XP, user requirements are expressed as scenarios or user stories. These are written on cards and the development team break them down into implementation tasks. These tasks are the basis of schedule and cost estimates. The customer chooses the stories for inclusion in the next release based on their priorities and the schedule estimates.
How does XP work? (1) ♦
♦
♦
The planning game
long-term plans only diffuse
Details in short-terms (some days)
customer is involved in every development cycle (some weeks)
tasks are handed out according to skills
Short development cycles
enforce decomposition into small tasks
minimize risk
better integration of customer
Simple Design
Design takes into account only short-term goals
Æ ♦
complexity is low
Sustainable amount of time
Large amounts of overtime are not considered acceptable (reduce
How does XP work? (2) ♦
Tests Unit tests: For implementation details, are developed by the developers themselves, are designed before development “Test-first development” enhances trust in system
♦
Refactoring
♦
allow for continuous changes in design Keep code simple and maintainable
Programming in pairs
always pair-programming changing pairs (daily) distributes knowledge within the team
How does XP work? (3) ♦
Collective ownership: Source code belongs to the team
♦
Continuous integration
♦
integration several times a day system is error-free every evening
Standards (Coding)
♦
everybody can make changes (consultation) no fixed responsibilities pairs work on all areas, no islands of expertise
is commonly accepted
Customer agent is part of the team
available (in case of questions) works out feature tests
