CCPDSDS- R Case Case St u d y and an d Sof t w are Pr Pr oces ocess s Tailor Tailor in g: Walker Royc Royce e I nsight nsight s Barr y Boehm CS 577a 577 a Lec Lectt ure ur e
Version 2.3
1
Outline • CCPDS-R Case Study – Pr oj ect ect Overview verv iew – Plans lan s an an d Schedu Schedules les – Resu esu lt s and Cr i t ical Su Su cces ccess s Fact act ors or s • Pr oces ocess Tail Tailor or ing in g • General ener al Bes Best Pr act act ices
Version 2.3
2
Managing Managing Succe ucce ssf ul I t e rat ive D evelopm ent Pr Pr oj e ct s A Semi nar on Sof Sof t w are Bes Bestt Pract ices Version 2.3 280 0 San Tom Tom as Exp Exp ressw ressw ay Sant a Clar Clar a, Ca Ca 950 51 (408) 496-3600
Version 2.3
3
Case St udy: CCPDS-R Proj ect Overview Charact erist ic CCPDS-R Dom ain Ground based C3 developm ent Size/ language 1.15M SLOCAda Average num ber of 75 people Schedule 75 m ont hs Process/ st andards DOD-STD-2167A I t erat ive development Environm ent Rat ional host DEC host DEC VMS targets Cont ract or TRW Cust om er USAF Cu rr en t st at u s Del iv er ed On -bu dget , On -sch edu le
Version 2.3
4
CCPDS-R Case St udy Overview • Organization • Subsystems/schedules • I teration plan/cont ent • Core m etr ics è Development progress (size w it h respect t o tim e) è Test pr ogress (evaluation crit eria wit h respect to tim e) è Soft w are architecture stabilit y (num bers of objects w ith respect t o tim e) è Subsystem stabilit y (SCOs w ith respect t o tim e) è Modularit y (SLOC: scrap w it h respect t o time) è Modularit y (SLOC % of t otal: scrap w it h respect to t ime) è Adaptability (rew ork w ith respect to t ime) è Matur ity (reliability w ith respect to tim e) è SCO volumes (r aw dat a by CSCI) è Cost/effort partitions by activity è Conclusions
Version 2.3
5
CCPDS-R Organization/Responsibilities Soft w are Proj ect Manager 2 people Chief Engineer 3 people
Admin 5 people
Major mi lestones Specifications Soft w are Engineerin g 10 people Demonstrations Metrics NAS CSCI Process ( SDP/ SSPM) Architecture
Version 2.3
WBS, cont ract Cost/ schedule
Applicat ions Soft w are 10-> 40 people SSV CSCI Architecture DCO CSCI CCO CSCI CMP CSCI TAS CSCI St andalone testing
6
Testing 5-> 20 people CM, CCB, t est bed s Environm ent/ resources Form al testing Str ing testing Tools/ scenarios
CCPDS-R Overview Common Subsystem PDS Subsystem STRATCOM Sub syst em
Proj ect
Size
Com m on months PDS months STRATCOM m ont hs Tot al months
Software effort:
Version 2.3
Schedule
353 KSLOC
60
254 KSLOC
32
552 KSLOC ___________
36
1159 KSLOC
75
75 people average
7
Com m on Subsystem Build Cont ent Primit ives and support soft w are
Architecture, t est scenarios, models
Crit ical t hread applications, displays
Mission algorit hms, non-crit ical thr ead applications, displays
Commu nications int erfaces, final test scenarios
Associate cont ractor i nt erface
Reliability and performance-critical component development
Version 2.3
Reliability and performance-critical component testing and maturation
8
Ada Pr ocess Model f or Lar ge- Syst em RAD
Version 2.3
9
Com m on Subsyst em Macropr ocess Developm ent Lif e Cycle Elabor at ion
Inception
Const ruct ion
Archit ect ure I t erat ions
Release I t erat ions
{ SSR 0
5
I PD R
PDR
10
15
CDR 20
25
Contr act award Architecture b aseline under change cont rol
Comp eti t ive design ph ase: • Architectur al prot otypes • Planning • Requirements analysis
Version 2.3
Early delivery of “alpha” capabilit y to user 10
Com m on Subsyst em Microprocesses 7.9 KSLOC
CDW
PD W
TOR
PDW: Prelimin ary Design Walkt hrough CDW: Crit ical Design Walkt hrough TOR: Turnov er Review
TOR
A1
43.5
CDW
PD W
TOR
TOR
Developm ent
A2
60.3
CDW
PD W
172.1
TOR
Test Turnover for configuration control
TOR
A3 PD W
62.7
CDW
TOR
TOR
A4 CDW
PD W
6.5
TOR
A5 PD W
CDW
TOR
35 3 SSR 0
Version 2.3
5
I PD R 10
CDR
PDR 15
11
20
25
30
34
Com m on Subsyst em Progr ess % 10 0 90
Plan
Designed/coded (% complete)
Actual
% of SLOC
80
KSLOC under baseline configuration control
10 0
70 60
75
50 40
50
30 20
25
10 5
10
15
20
25
30
35
Contract month I PDR
PDR
• CDR pr ogr ess: Soft w are design Code developm ent Baseline under change cont rol Form al t est Performance assessment
Version 2.3
10
15
20
25
30
35
Contract month
CDR
Tradition al Approach
CCPDS-R App roach
Complete 10% Negligible 0% Modeling
Complete 94 % 47 % 12 % 80% of operational software demonstrated
12
Com m on Subsyst em Adapt abilit y n
Archit ect ure fir st – I nt egration dur ing t he design phase – Demonst rat ion-based evaluation
n
Conf igur at ion baseline change met ri cs:
40 Design Changes
Hours 30 Change 20
Maintenance Changes and ECPs Implementation Changes
10 Project Development Schedule
15
Version 2.3
20
13
25
30
35
40
Som e Gener al Conclu sions n
Com m on subsyst em subsidized: è
I n t e r n a l r e u se
è
Comm on process/ t ools
1
Normalized $ / SLOC
è
Overall systems engineering
è
Cost / SLOC: Com m on PDS
0.5
$X/ lin e $.40X/ lin e STRATCOM $.33X/ lin e 0 Common
n
STRATCOM
Product ivit y/ qualit y im proved w it h each subsyst em . Com m on PDS STRATCOM –
n
PDS
This is the real in dicator o f a matur e (level 3/ level 4) process.
Personnel volat ilit y w as low on t he comm on subsystem. –
It was much hi gh er o n the PDS and STRATCOM subsystem s.
Version 2.3
14
General Con clusions (Cont inued) • CCPDS-R adhered to 2167A – Process could have been much more efficient – Early (init ially unplanned) delivery (m onth 30) of a useful increment • Moderate contractual requirements volatility
•
– Heavy non-ECP requirements/design volatilit y – Only 1 major ECP (complete overhaul of t he input message set) – Contractor/customer/user rapport avoided an inefficient contracts bureaucracy Personnel skill: – Strong proj ect management , planning, and archit ecture teams – Cooperative custom er/user – Average application/t est t eam
Version 2.3
15
CCPDS-R Soft w are Technology Thrusts CCPDS-R Approach
Curr ent Thru st
E
· · · ·
Size
· Reuse, COTS
·
· Object-oriented · Higher level languages · CASE to ol s
· · ·
· Distributedmiddleware
·
· · · ·
· · · ·
P
I ntegrated tools Open system s Hardware performance Automation
I terative development Process maturity models Architecture-first Acquisition r eform
· Training
Version 2.3
· · · ·
·
DEC/ Rational/ custom tools VAX/ DEC depen dent Several VAX famil y upg rades Custom- developed change m anagement syst em, met rics tools, code audit ors Comm on architectur e primit ives, tools, processes across all subsystems Message-based, obj ect- orient ed archit ectu re 100% Ada Custom autom atic code generat ors for architectur e, message I / O, display form at source code generation Early in vestm ent in NAS development for reuse across mult iple subsystem s Demonstr ation, mul ti ple builds, early delivery Level 3 process prior to SEI CMM definit ion Executable archit ectu re baseline w it h CM at PDR Excellent customer/ contr actor/ user teamwork , highly t ailored 2167A for it erative development Mostl y OJT and int ernal ment oring
16
CCPDS- R MBASE Model s Success Models – Reint erpr et ed DOD-STD-216 7a; u sers in volv ed – Aw ard fee flow dow n t o perform ers • Produ ct Models – Domain m odel and architect ure – Message-passing m iddl ew are ( UNAS) • Process Models – Ada process m odel and t oolset – I ncrem ental builds; early delivery • Propert y Models – COCOMO cost & schedule – UNAS - based performance modeling – Ext ensive progress and qualit y m etr ic tools •
Version 2.3
17
Agend a •
•
•
•
•
Motivation – The search for soft w are best practices – Soft w are economics Technology overview s – I t erative development – Architectures – SEI CMM and I SO 90 00 Soft w are process – The Rat ional Obj ector y pr ocess – An organizational process – Tailoring t he process – The acquisit ion pr ocess Exper ience and result s – General observat ions – Case St udies and r eal-w orld metrics Conclusions
Version 2.3
Tailorin g t he pr ocess â
Present t he range of applicability â
Same invariant spirit / policy
â
Different priorities
â
Different im plement ations
18
Classif yi ng Syst em s Higher Technical Com plexi t y
An average softw are project: 5-10 people 10-15 month duration 3-5 external interf aces Some unknow ns, risks
- Embedded, real-time, distributed, fault-tolerant - Custom, u nprecedent ed, archit ectu re r eengineeri - High perform ance DoD Weapon System Telecom Switch Commercial Compiler Embedded Automotive Software Case Tool (Rose, SoDA)
Lower Management Complexity -
Smal l Scale Informal Single stak eholder “Products”
Large-Scale Organizational/ Entit y Simulation
Small Scientif ic Simulation
Business Spreadsheet
I S Application Distributed Objects (Order Entry)
Enterpri se I S (Family of I S Applications)
DoD MI S System
I S Application GU I / R DB (Order Entry)
Lower Technical -Complexity Mostly 4GL, or component -based - Application reengineering - Interactive performance
Version 2.3
National ATC System
19
Higher Management Complexity -
Large Scale Contractual Many stake holders “Projects”
Pr ocess Tail or in g Higher Techn ical Com plexi t y • • • •
More emphasis on dom ain experience Longer inception/ elaboration phase More iterat ions, risk management Less predi ctable cost s/ schedules
Lower Management Complexity
Higher Management Complexity • • • •
More emphasis on m anagement perspective More process form ality/ ceremony More emphasis on teamwork and win/ w in Longer I nception/ elaboration
Low er Technical Comp lexit y • • • • Version 2.3
More emphasis on existing assets/ know ledge base Shorter inception/ elaboration phase Fewer iterations More pr edictable costs/ schedules 20
UNI X vs. Windows Higher Technical Complexity
DoD Weapon System
Generally Microsoft Window s or Window s NT
Telecom Switch Commercial Compiler
Embedded Automotive Software Case Tool
(Rose, SoDA)
Lower Management Complexity
Large-Scale Organizational/ Entit y Simulation
Small Scientif ic Simulation I S Application Distributed Objects (Order Entry)
Enterpr ise I S (Family of I S Applications)
I S Application GU I / R DB (Order Entry) Business Spreadsheet
Lower Technical Complexity
Version 2.3
National ATC System
21
Higher Management Complexity
DoD MI S System
Generally UNI X and ot her non-Windows platforms
Pr oduct s v s. Pr oj ect s •
Emph asis and focus are dif ferent . – Product s: Mark et- const rained, small t eams, driv en by t echnical challenges
Varyin g levels are fr eely t raded off w it h cost and time-to-market
• Usability • Reusability • Portability • Adaptability
– Proj ect s: Cont ract const rained, large t eams, driven by management challenges
Fix ed levels are requir ed at fix ed cost s and schedules
• Completeness • Performance • Reliability
•
The same pro cess th emes apply. – How ever, priorit ies are diff erent .
Version 2.3
22
Pr oduct s vs. Projects • Top 10 discrimi nati ons of success/ failur e:
Projects
Products 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
1. Archit ect ure t eam skill Obj ect -ori ented t echnology 2. 3. Programm ing t eam skill 4. I t erat ive development Change m anagem ent rigor 5. 6. User/ cust omer focus 7. Environm ent int egrat ion 8. Languages/ st andards 9. Management t eam skill 10. Megaprogramming
Version 2.3
23
I t erative development Archit ect ure t eam skill Management t eam skil l Acquisition process/ rapport Megaprogramming Change m anagem ent rigor Envir onment int egrat ion Languages/ st andards Obj ect -ori ented t echnology Programm ing t eam skill
Product Discrim inat ors Architect ure/ programm ing t eam skills are paramount . – I nnovat ion and heroic effor t s are crit ical • Managem ent skil l is less im port ant – There are few constr aint s. Tim e-t o-m arket , fun ction, qualit y, and cost are negot iable. – Tradeoff s can be decided upon w it hin comm on corporat e goals. • Obj ect -or ient ed technology is key t o t echni cal success. – Reuse, adaptabil it y and port abilit y are necessit ies • Acquisition process/ rapport is generally unim port ant. – User/ cust omer focus is st ill vit al – The mark et i s m oldable; cust omer s appears aft er development. •
Version 2.3
24
Project Discriminators I t erati ve development and acquisit ion process are key. – R&D ( resolving un know ns in r equirements, design, and t echnology) i s separat e from pr oduct ion (development of useful increment s). • Archit ect ure team skill is very import ant. – # 1 driver of t echnical qualit y, feasibilit y, and applicat ion production • Management skill is very im port ant. – Many const raint s (cost , schedule, funct ion, qualit y) ; cont inuous negot iati on and t eam buil ding • Applicat ions team skill is less import ant. – A good archit ect ur e can be impl ement ed by an average team. •
Version 2.3
25
Best Pr act ices Cr oss Reference 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 17. 19. 20. 26.
Make Quality # 1. High-qualit y soft w are is possible. Give pr oduct s t o cust om ers early. Det ermine t he problem before w rit ing the requirement s. Evaluat e design alt ernat ives. Use an appr opri at e process m odel. Use different languages for dif ferent phases. Minim ize int ellect ual distance. Put t echniques before t ools. Get it right before you make it fast er. I nspect code. Good m anagem ent is m ore im port ant t han good t echnology. People are t he hey t o success. Plan t o t hrow one aw ay. Design for change. Design w it hout docum entat ion is not design. Don’t test your ow n soft w are.
Version 2.3
26
Our Preferred Wording 1. Define qualit y comm ensurat e w it h your objectives 2. Make qualit y assurance a team goal, not a separate di scipline 3. Give produ cts to custom ers early. 4. Determi ne the problem before wr it ing the requirements. 5. Evaluat e design alt ernat ives. 6. Tailor t he process t o th e scale/domain/complexit y of t he proj ect 7. Minimize t he number of life-cycle represent ation f orm ats. 8. Minimize intellectu al dist ance (use an obj ect oriented m ethod) . 9. Put t echniques before t ools. 10. Get it w orking before you make it f aster. 11. Use inspect ions j udiciously 12. Good management is more import ant t han good technology 13. People are t he k ey t o success. 17. Encourage experi ment ati on and exposure of design issues. 19. Design f or change. 20. Build self-document ing pr oduct s rather t han self-serving documents. 26. Plan and execut e a test pr ogram f rom day 1.
Version 2.3
27
Recurring Themes of Success n
Customer-user-contractor teamwork is essential – Relationships are non-adversarial.
n
Managers are perform ers. – They author t heir ow n plans.
n
Requirement s/designs are fluid and t angible. – Mat urit y evolves w here designs are “ guilty unt il proven innocent . – Real issues are sur faced and r esolved syst emat ically. – Change m anagement overri des change avoidance.
n n
CM/QA is everyone’s j ob, not a separat e discipline. Perform ance issues arise early in t he lif e cycle. – “ Something” is performing.
n
Low att rit ion of good people is a sign of success.
Version 2.3
28
Recommendat ions t o Buyers n
n
Try t o acquir e a good product and have your cont ract or make a good profit Inst all an on-line acquisit ion environm ent : – Demand delivery of execut able evolving product versions. Ù
Get users involved early and continuously.
– Add value t hrough your ow n engineering support . Ù
Do your ow n m etr ics analysis. Ù Const ru ct scenarios. n
Avoid att rit ion and adversarial relationships.
n
St rive for 80% solut ion early, then evolve tow ard 100%. – Separate the “need” from the “ w ant.” Version 2.3
29
Recommendations t o Contract ors n
Iterat ive development exposes import ant t rends early. – Good early perf orm ance breeds cont inuous success. – Poor early perform ance is almost impossible to t urn around. Ù
Get the right management team. Ù Get t he right architecture team. n
Provide adequat e capital environments. – I terative development is much m ore (comput er) resource intensive – Opportunit ies for automation must be exploit ed.
n
Enable round-trip engineering. – Int egrat ed environment s and compat ible tools.
Version 2.3
30
Rat ional Process Summary Development Life Cycle I ncept ion
Elaborati on
Constru ction
Transition Development
Objective
Identify Revenue opportunity
Risk resolution
Product
Business case
Architecture an d Production plan
Useful increment s Version updates
Cost estim ate
Prototyping Object point s
Early design Function points
Post-architecture Maintenance Sour ce lin es Annual change traffic
Risk Focus
Technical and economic feasibility
Schedule
Cost
Version 2.3
31
User satisfaction
Deployment perturbations
The I mport ance of Automation n
Process maturit y depends on capable environment s. – Met rics must be aut omatically collect ed by the environm ents. Ù
I t’s the best w ay softw are engineers w ill accept them. – Change management must be aut omat ed and enforced. Ù I t’s the best w ay to mange mult iple iterations. Ù I t’s the best w ay to enable change freedom. Ù Change is the fundamental primit ive of it erative development. – An archit ect ure-first and demonst ration-based process is enabled by off-the-shelf architecture components and middleware. Ù I t’s the best w ay to remain hardw are/topology-independent. – Tools must be pre-int egrated. Ù I t ’s t he best w ay to m aintain consistency and traceability . – Test ing/documentation m ust be mostly automated. Ù I t’s the best w ay to attain change freedom Version 2.3
32
Making a Posit ive Econom ic I mpact Cost = (Environm ent)* (Size) (Process) n
Environment s are t he key t o reduced “ coefficients.” – More automation and integration – Change management and change fr eedom
n
Obj ect s technology is t he key t o reduce “Size.” – Reuse, off-t he-shelf product s, autom atic code generation – Line-of-b usiness archit ect ures
n
I terative development is the key t o reduced “exponents.” – Architecture-driven, continuous integration, tangible quality/progress
Maintain a balance of emphasis on all 3 thr usts.
Version 2.3
33
Soft w are Management Balance is Key Too l it t l e/ f ew Burnout and attrition Inadequate vision Inadequate ROI
People Requirements detail Institutionalization
Inadequate quality
Schedul e/ cost
Inadequate legacy
Documentation
Version 2.3
34
Too m u ch / m a ny I nefficiency and bureaucracy I nefficiency of change I nefficiency of change I nefficiency of r esour ces Diluted product focus
