David Noble
[email protected] Sr. Solutions Architect, Exist Global
Testing in Java San Gabriel Valley Java Users Group http://www.sgvjug.org/ June 25, 2007
Goal Motivate excellence in testing with concepts, practices, and tools
Goal Motivate excellence in testing with concepts, practices, and tools
2
Scope requirements
continuous integration
Testing concepts, practices, tools
quality assurance
static code
Scope requirements
continuous integration
Testing concepts, practices, tools
debugging
quality assurance
static code analysis
profiling 3
Agenda Test-Driven Development Terminology Tools Application Discussion
Agenda Test-Driven Development Terminology Tools Application Discussion
4
System Concept System Requirements System Design Detailed Design Implementation
g n
System Concept System Requirements System Design Detailed Design Implementation
g n i t s e T http://www.flickr.com/photos/h2oalchemist/342496829/
5
Demo A Taste of TDD
Demo A Taste of TDD
Test-Driven Development Add test Fail
Make small change Pass Pass
Test-Driven Development Add test Fail
Make small change Pass Pass
Commit changes See also: http://www.agiledata.org/essays/tdd.html
Test-Driven Development Add test Fail
Fix bug Pass
7
Test-Driven Development Add test Fail
Fix bug Pass
Commit changes 8
Test-Driven Development Refactor Code
Pass
Commit changes Add test Fail
Make small change Pass
Pass
Test-Driven Development Refactor Code
Pass
Commit changes Add test Fail
Make small change
Pass
Pass
Commit changes
9
Test-Driven Development Small steps Frequent check-ins
Test-Driven Development Small steps Frequent check-ins
10
Test-Driven Development Deeper than “test first” Design for testability
Test-Driven Development Deeper than “test first” Design for testability
11
Test-Driven Development Unit tests as example code Interface before implementation implementation Function drives form
Test-Driven Development Unit tests as example code Interface before implementation implementation Function drives form
12
Test-Driven Development Loose coupling Services with clear responsibilities Mock objects
Test-Driven Development Loose coupling Services with clear responsibilities Mock objects
13
Taste the soup!
Taste the soup! 14 http://flickr.com/photos/basak/227480302/
Intent Requirements, Architecture, Design
Reality Implementation, Deployment
Intent
Reality
Requirements, Architecture, Design
Implementation, Deployment
15
Intent Requirements, Architecture, Design
Reality
! p a G
Implementation, Deployment
Intent Requirements, Architecture, Design
Reality
! p a G
Implementation, Deployment
16
Intent Requirements, Architecture, Design
Accountability Reality Testing, QA
Implementation, Deployment
Intent Requirements, Architecture, Design
Accountability Reality Testing, QA
Implementation, Deployment
17
Intent Requirements, Architecture, Design
Accountability Reality Testing, QA
y t i ! l k a c e R h e C
Implementation, Deployment
Intent Requirements, Architecture, Design
Accountability Reality Testing, QA
y t i ! l k a c e R h e C
Implementation, Deployment
18
Terminology QA – quality assurance Do the right things Do things right Ensure quality standards
Terminology QA – quality assurance Do the right things Do things right Ensure quality standards
19
Terminology Test – success criteria Is it right? Is it done?
Terminology Test – success criteria Is it right? Is it done?
20
Terminology Test plan Test case Test suite Test report Test coverage
Terminology Test plan Test case Test suite Test report Test coverage
21
Terminology Unit test Functional test Integration test System test Acceptance test
Terminology Unit test Functional test Integration test System test Acceptance test Regression test 22
Terminology Non-functional Non-functional testing Performance test Load test Soak test
Terminology Non-functional Non-functional testing Performance test Load test Soak test
23
Terminology Continuous integration
Terminology Continuous integration
24
Tools
Mock Objects jMock, EasyMock Unit Tests Build IDE
JUnit, TestNG Ant, Maven Eclipse, NetBeans
Tools
Mock Objects jMock, EasyMock Unit Tests Build IDE
JUnit, TestNG Ant, Maven Eclipse, NetBeans
25
Tools
Web Apps
Selenium, HttpUnit
JEE Servers
Cactus
Databases
DBUnit
Tools
Web Apps
Selenium, HttpUnit
JEE Servers
Cactus
Databases
DBUnit
26
Tools
Acceptance Performance Coverage
Selenium, FitNesse, Stiq JMeter, Eclipse TPTP Cobertura, Emma
Tools
Acceptance Performance Coverage
Selenium, FitNesse, Stiq JMeter, Eclipse TPTP Cobertura, Emma
27
Tools
Continuous Integration
Cruise Control, Continuum, Hudson
Tools
Continuous Integration
Cruise Control, Continuum, Hudson
28
Tools Continuous Integration Acceptance Performance Coverage
g n i t s e
Web Apps
Cruise Control, Continuum, Hudson Selenium, FitNesse, Stiq JMeter, Eclipse TPTP Cobertura, Emma Selenium, HttpUnit
JEE Servers
Cactus
Databases
DBUnit
Mock Objects
jMock, EasyMock
Tools Continuous Integration Acceptance Performance Coverage
g n i t s e t t i n U
Web Apps
Cruise Control, Continuum, Hudson Selenium, FitNesse, Stiq JMeter, Eclipse TPTP Cobertura, Emma Selenium, HttpUnit
JEE Servers
Cactus
Databases
DBUnit
Mock Objects Unit Tests Build IDE
jMock, EasyMock JUnit, TestNG Ant, Maven Eclipse, NetBeans 29
Demo Tools
Demo Tools
Resources: SGV JUG Web Site http://sgvlug.net/mailman/listinfo/java-sig
Mailing List (Hosted by SGV Linux Users Group) http://sgvlug.net/mailman/listinfo/java-sig
[email protected]
Resources: SGV JUG Web Site http://sgvlug.net/mailman/listinfo/java-sig
Mailing List (Hosted by SGV Linux Users Group) http://sgvlug.net/mailman/listinfo/java-sig
[email protected]
31
Resources: TDD Articles http://www.agiledata.org/essays/tdd.html http://en.wikipedia.org/wiki/Test_driven_development http://www.testdriven.com/modules/xoopsfaq/ http://www.artima.com/intv/testdriven.html http://www.google.com/search?hl=en&q=test+driven+development (really!)
Resources: TDD Articles http://www.agiledata.org/essays/tdd.html http://en.wikipedia.org/wiki/Test_driven_development http://www.testdriven.com/modules/xoopsfaq/ http://www.artima.com/intv/testdriven.html http://www.google.com/search?hl=en&q=test+driven+development (really!)
32
Resources: IDE & Build Tools Eclipse IDE http://www.eclipse.org/
NetBeans IDE http://www.netbeans.org/
Ant http://ant.apache.org/
Maven 2 http://maven.apache.org/ http://www.devzuz.com/web/guest/products/resources#BBWM
Maven & Eclipse
Resources: IDE & Build Tools Eclipse IDE http://www.eclipse.org/
NetBeans IDE http://www.netbeans.org/
Ant http://ant.apache.org/
Maven 2 http://maven.apache.org/ http://www.devzuz.com/web/guest/products/resources#BBWM
Maven & Eclipse http://m2eclipse.codehaus.org/
Maven & Netbean http://maven.apache.org/guides/mini/guide-ide-netbeans/guide-ide-netbeans.html
33
Resources: Unit Testing JUnit Framework http://junit.org/
JUnit FAQ http://junit.sourceforge.net/doc/faq/faq.htm
JUnit with Ant http://ant.apache.org/manual/OptionalTasks/junit.html http://ant.apache.org/ http://ant.apache.org/manual/OptionalTasks/junitreport.html manual/OptionalTasks/junitreport.html
JUnit with Maven http://maven.apache.org/plugins/maven-surefire-plugin/
TestNG Framework
Resources: Unit Testing JUnit Framework http://junit.org/
JUnit FAQ http://junit.sourceforge.net/doc/faq/faq.htm
JUnit with Ant http://ant.apache.org/manual/OptionalTasks/junit.html http://ant.apache.org/ http://ant.apache.org/manual/OptionalTasks/junitreport.html manual/OptionalTasks/junitreport.html
JUnit with Maven http://maven.apache.org/plugins/maven-surefire-plugin/
TestNG Framework http://testng.org/doc/
Jester – Mutation Testing http://jester.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-jester/
Resources: Mock Objects Mock Objects Blog http://mockobjects.com/
EasyMock Library http://easymock.org/
jMock Library http://jmock.org/
34
Resources: Mock Objects Mock Objects Blog http://mockobjects.com/
EasyMock Library http://easymock.org/
jMock Library http://jmock.org/
35
Resources: Unit & Functional XmlUnit Library http://xmlunit.sourceforge.net/
DBUnit Library http://dbunit.sourceforge.net/
HttpUnit Library http://httpunit.sourceforge.net/
Cactus for JEE Server Code http://jakarta.apache.org/cactus/
Cargo for JEE Server Code
Resources: Unit & Functional XmlUnit Library http://xmlunit.sourceforge.net/
DBUnit Library http://dbunit.sourceforge.net/
HttpUnit Library http://httpunit.sourceforge.net/
Cactus for JEE Server Code http://jakarta.apache.org/cactus/
Cargo for JEE Server Code http://cargo.codehaus.org/
36
Resources: Test Coverage Cobertura http://cobertura.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-cobertura/
Cobertura with Ant http://cobertura.sourceforge.net/anttaskreference.html
Cobertura with Maven http://mojo.codehaus.org/cobertura-maven-plugin/usage.html
Emma http://emma.sourceforge.net/
Resources: Test Coverage Cobertura http://cobertura.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-cobertura/
Cobertura with Ant http://cobertura.sourceforge.net/anttaskreference.html
Cobertura with Maven http://mojo.codehaus.org/cobertura-maven-plugin/usage.html
Emma http://emma.sourceforge.net/
37
Resources: Perf. & Load JUnitPerf http://clarkware.com/software/JUnitPerf.html
JMeter http://jakarta.apache.org/jmeter/
Eclipse TPTP (Test & Performance Tools Platform) http://www.eclipse.org/tptp/
Netbeans Profiler http://profiler.netbeans.org/
Resources: Perf. & Load JUnitPerf http://clarkware.com/software/JUnitPerf.html
JMeter http://jakarta.apache.org/jmeter/
Eclipse TPTP (Test & Performance Tools Platform) http://www.eclipse.org/tptp/
Netbeans Profiler http://profiler.netbeans.org/
38
Resources: Acceptance Tests Selenium for Web Apps http://www.openqa.org/selenium/ http://www.openqa.org/selenium-ide/download.action
Automating Selenium (“Remote Control”) http://www.openqa.org/selenium-rc/ http://wiki.openqa.org/display/SRC/Selenium-RC+and+Continuous+Integration
Selenium & Maven http://raibledesigns.com/rd/entry/selenium_plugin_for_maven
FitNesse
Resources: Acceptance Tests Selenium for Web Apps http://www.openqa.org/selenium/ http://www.openqa.org/selenium-ide/download.action
Automating Selenium (“Remote Control”) http://www.openqa.org/selenium-rc/ http://wiki.openqa.org/display/SRC/Selenium-RC+and+Continuous+Integration
Selenium & Maven http://raibledesigns.com/rd/entry/selenium_plugin_for_maven
FitNesse http://fitnesse.org/
Stiq http://storytestiq.sourceforge.net/
39
Resources: Cont. Integration Cruise Control http://cruisecontrol.sourceforge.net/
Continuum http://maven.apache.org/continuum/
Hudson https://hudson.dev.java.net/
Resources: Cont. Integration Cruise Control http://cruisecontrol.sourceforge.net/
Continuum http://maven.apache.org/continuum/
Hudson https://hudson.dev.java.net/
40
Resources: App used in Demo Open Source Medical Record System http://www.openmrs.org/
Prerequisites: SVN, MySQL 5, Tomcat 5 http://subclipse.tigris.org/update_1.2.x http://mysql.org/ http://tomcat.apache.org/
SVN Repository http://svn.openmrs.org/openmrs/branches/alpha
Installation
Resources: App used in Demo Open Source Medical Record System http://www.openmrs.org/
Prerequisites: SVN, MySQL 5, Tomcat 5 http://subclipse.tigris.org/update_1.2.x http://mysql.org/ http://tomcat.apache.org/
SVN Repository http://svn.openmrs.org/openmrs/branches/alpha
Installation Edit build.properties and properties.xml to set tomcat.user/password, or add test/test to $CATALINA_HOME/conf/tomcat-users.xml file. Edit build.properties and/or properties.xml to set tomcat.home file. Comment out creation of "test" MySQL user in schema if it already exists. Default username/password is admin/test.
Thank you To all who participated!
41
Thank you To all who participated!
David Noble
[email protected] Sr. Solutions Architect, Exist Global
Testing in Java San Gabriel Valley Java Users Group http://www.sgvjug.org/
David Noble
[email protected] Sr. Solutions Architect, Exist Global
Testing in Java San Gabriel Valley Java Users Group http://www.sgvjug.org/ June 25, 2007
My background: Mostly C & Perl from 1990 – 1997 Mostly Java since then. Telemetry processing, mission ops, EIS @ JPL. BPM, workflow, assorted biz apps.
1
Goal Motivate excellence in testing with concepts, practices, and tools
2
Goal falls within the larger context of pursuing excellence in software development. development.
2
Scope requirements
continuous integration
Testing concepts, practices, tools
debugging
quality assurance
static code analysis
profiling 3
3
Agenda Test-Driven Development Terminology Tools Application Discussion
4
4
System Concept System Requirements System Design Detailed Design Implementation
g n i t s e T http://www.flickr.com/photos/h2oalchemist/342496829/
5
In traditional software development, testing is usually at the end of the schedule. If development runs late, the testing often suffers.
5
Demo A Taste of TDD
6
Test-Driven Development Add test Fail
Make small change Pass Pass
Commit changes See also: http://www.agiledata.org/essays/tdd.html
7
7
Test-Driven Development Add test Fail
Fix bug Pass
Commit changes 8
8
Test-Driven Development Refactor Code
Pass
Commit changes Add test Fail
Make small change Pass
Pass
Commit changes
9
9
Test-Driven Development Small steps Frequent check-ins
10
10
Test-Driven Development Deeper than “test first” Design for testability
11
11
Test-Driven Development Unit tests as example code Interface before implementation Function drives form
12
12
Test-Driven Development Loose coupling Services with clear responsibilities Mock objects
13
13
Taste the soup! 14 http://flickr.com/photos/basak/227480302/
An old man calls the waiter over to his table. "What would you like, sir?" asks the waiter. "Taste the soup," he says. "Is the soup too cold?" asks the waiter. "Taste the soup," says the old man. "Is it too salty?" asks the waiter. "Taste the soup!" says the old man. "Is there a fly in it?" asks the waiter. "Taste the soup!" the old man insists. The waiter looks down: "There's no spoon!" "Aha!" exclaims the old man.
14
Intent Requirements, Architecture, Design
Reality Implementation, Deployment
15
15
Intent Requirements, Architecture, Design
Reality
! p a G
Implementation, Deployment
16
16
Intent Requirements, Architecture, Design
Accountability Reality Tes Te sti tin ng, QA
Implem Impl emen enta tati tion on,, Deployment
17
17
Intent Requirements, Architecture, Design
Accountability Reality Tes Te sti tin ng, QA
y t i l k ! a e e c R h C
Implem Impl emen enta tati tion on,, Deployment
18
18
Terminology QA – quality assurance Do the right things Do things right Ensure quality standards
19
Miriam-Webster: “a program for the systematic monitoring and evaluation of the various aspects of a project, service, or facility to ensure that standards standards of quality are being met”
19
Terminology Test – success criteria Is it right? Is it done?
20
20
Terminology Test plan Test case Test suite Test report Test coverage
21
Test plan: document Test case: implementation implementation of specific specifi c test. Test suite: collection of test cases. Test report: results of running the tests. Test coverage: description of how much code (and which code) was exercised by tests.
21
Terminology Unit test Functional test Integration test System test Acceptance test Regression test 22
Unit test: typically at the granularity granularity of a class. Functional test: a feature or use case. Integration test: multiple modules. System test: all the modules. Acceptance Acceptance test: defined by user/customer. user/customer.
22
Terminology Non-functional testing Performance test Load test Soak test
23
Non-functional testing: the “-ilities”. Performance test: how fast. Load test: how many. Soak test: how long.
23
Terminology Continuous integration
24
This is an alternative alternative to “big bang integration”, otherwise known as “integration hell”.
24
Tools
Mock Objects jMock, EasyMock Unit Tests Build
JUnit, TestNG Ant, Maven
IDE Eclipse, NetBeans
25
25
Tools
Web Apps
Selenium, HttpUnit
JEE Servers
Cactus
Databases
DBUnit
26
HttpUnit is better suited to situations where you want to examine the DOM at a low level. If you're looking for convenience, and don't need to get deep into the details, look at Selenium instead. i nstead.
26
Tools
Acceptance Performance Coverage
Selenium, FitNesse, Stiq JMeter, Eclipse TPTP Cobertura, Emma
27
Selenium IDE, a Firefox plugin, lets users or product managers managers create their own scenarios. Selenium is also useful during development, to quickly fast forward through actions to get the web application into a desired state.
27
Tools
Continuous Integration
Cruise Control, Continuum, Hudson
28
28
Tools Continuous Integration Acceptance Performance Coverage
g n i t s e t t i n U
Web Apps
Cruise Control, Continuum, Hudson Selenium, FitNesse, Stiq JMeter, Eclipse TPTP Cobertura, Emma Selenium, HttpUnit
JEE Servers
Cactus
Databases
DBUnit
Mock Objects Unit Tests Build IDE
jMock, EasyMock JUnit, TestNG Ant, Maven Eclipse, NetBeans 29
The items in green aren't really a stack, just a collection of technology-specific libraries that can be useful for creating test cases.
29
Demo Tools
Be sure to design Selenium tests so they can be repeatable. Either delete data when they are done, or always start from a fixed known state.
30
Resources: SGV JUG Web Site http://sgvlug.net/mailman/listinfo/java-sig
Mailing List (Hosted by SGV Linux Users Group) http://sgvlug.net/mailman/listinfo/java-sig
[email protected]
31
31
Resources: TDD Articles http://www.agiledata.org/essays/tdd.html http://en.wikipedia.org/wiki/Test_driven_development http://www.testdriven.com/modules/xoopsfaq/ http://www.artima.com/intv/testdriven.html http://www.google.com/search?hl=en&q=test+driven+development (really!)
32
32
Resources: IDE & Build Tools Eclipse IDE http://www.eclipse.org/
NetBeans IDE http://www.netbeans.org/
Ant http://ant.apache.org/
Maven 2 http://maven.apache.org/ http://www.devzuz.com/web/guest/products/resources#BBWM
Maven & Eclipse http://m2eclipse.codehaus.org/
Maven & Netbean http://maven.apache.org/guides/m http://maven.apache.org/guides/mini/guide-ide-netbeans/g ini/guide-ide-netbeans/guide-ide-netbeans.html uide-ide-netbeans.html
33
33
Resources: Unit Testing JUnit Framework http://junit.org/
JUnit FAQ http://junit.sourceforge.net/doc/faq/faq.htm
JUnit with Ant http://ant.apache.org/manual/OptionalTasks/junit.html http://ant.apache.org/manual/OptionalTasks/junitreport.html
JUnit with Maven http://maven.apache.org/plugins/maven-surefire-plugin/
TestNG Framework http://testng.org/doc/
Jester – Mutation Testing http://jester.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-jester/
34
34
Resources: Mock Objects Mock Objects Blog http://mockobjects.com/
EasyMock Library http://easymock.org/
jMock Library http://jmock.org/
35
35
Resources: Unit & Functional XmlUnit Library http://xmlunit.sourceforge.net/
DBUnit Library http://dbunit.sourceforge.net/
HttpUnit Library http://httpunit.sourceforge.net/
Cactus for JEE Server Code http://jakarta.apache.org/cactus/
Cargo for JEE Server Code http://cargo.codehaus.org/
36
36
Resources: Test Coverage Cobertura http://cobertura.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-cobertura/
Cobertura with Ant http://cobertura.sourceforge.net/anttaskreference.html
Cobertura with Maven http://mojo.codehaus.org/cobertura-maven-plugin/usage.html
Emma http://emma.sourceforge.net/
37
37
Resources: Perf. & Load JUnitPerf http://clarkware.com/software/JUnitPerf.html
JMeter http://jakarta.apache.org/jmeter/
Eclipse TPTP (Test & Performance Tools Platform) http://www.eclipse.org/tptp/
Netbeans Profiler http://profiler.netbeans.org/
38
38
Resources: Acceptance Tests Selenium for Web Apps http://www.openqa.org/selenium/ http://www.openqa.org/selenium-ide/download.action
Automating Selenium (“Remote Control”) http://www.openqa.org/selenium-rc/ http://wiki.openqa.org/display/SRC/S http://wiki.openqa.org/display/SRC/Selenium-RC+and+Co elenium-RC+and+Continuous+Integration ntinuous+Integration
Selenium & Maven http://raibledesigns.com/rd/entry/selenium_plugin_for_maven
FitNesse http://fitnesse.org/
Stiq http://storytestiq.sourceforge.net/
39
39
Resources: Cont. Integration Cruise Control http://cruisecontrol.sourceforge.net/
Continuum http://maven.apache.org/continuum/
Hudson https://hudson.dev.java.net/
40
40
Resources: App used in Demo Open Source Medical Record System http://www.openmrs.org/
Prerequisites: SVN, MySQL 5, Tomcat 5 http://subclipse.tigris.org/u http://subclipse.tigris.org/update_1.2.x pdate_1.2.x http://mysql.org/ http://tomcat.apache.org/
SVN Repository http://svn.openmrs.org/openmrs/branches/alpha
Installation Edit build.properties and properties.xml to set tomcat.user/password, or add test/test to $CATALINA_HOME/conf/tomcat-users.xml file. Edit build.properties and/or properties.xml to set tomcat.home file. Comment out creation of "test" MySQL user in schema if it already exists. Default username/password is admin/test.
41
41
Thank you To all who participated!
42