X36SSP
CMM + metriky
8. přednáška
Ing. Martin Molhanec
ČVUT – FEL
K13113
CMM
Capability Maturity Model
• Model vznikl z potřeby hodnotit pro ministerstvo
obrany USA softwarové firmy při výběrových
řízeních na státní zakázky v počátku osmdesátých
let. Byl vytvořen pracovníky Institutu
softwarového inženýrství Carnegie Mellon
univerzity v Pitsburgu.
• Vžil se pod označením CMM (Capability Maturity
Model) nebo také pod označením SW-CMM,
protože později začaly vznikat jeho modifikace
pro různé jiné oblasti (PM-CMM pro projektové
řízení a další). Model definuje firemní procesy v
softwarových firmách do pěti úrovní.
Capability Maturity Model provides a framework from which
a process for large, complex software efforts can be
defined
Maturity
• The Capability Maturity Model (CMM) distinguishes between immature and
mature software organizations.
• Immature software organizations are typically reactionary and have little
understanding as to how to successfully develop software.
• Mature software organizations understand the development process, which
enables them to judge the quality of the software products and the process
that produces them. Mature software organizations have a higher success
rate and a lower overall cost of software across the entire life of a software
product than do immature software organizations.
• The CMM has been adopted by hundreds of organizations worldwide who want
to improve the way they develop software.
Level 5: Optimized
Work the measures
Level 4: Managed
Measure the work
Level 3: Defined
Work the plan
Level 2: Repeatable Plan the work
Level 1: Initial
Worship the hero
The CMM defines five maturity levels, evolutionary plateaus toward achieving a mature
software process, that an organization can attain with respect to the software process.
Source: Software Engineering Institute
CMM defines issues to be addressed
The true value of the CMM is that it defines the
issues that must be addressed by a software process
to achieve each maturity level.
These issues are called key process areas (KPA).
Level 5: Optimizing
Defect Prevention
Technology change management
Process change management
Level 4: Managed
Quantitative process management
Software quality management
Level 3: Defined
Level 2: Repeatable
Organization process focus
Organization process definition
Training program
Integrated software management
Software product engineering
Intergroup coordination
Peer reviews
Requirements management
Software project planning
Level 1:
Initial
Software project tracking and over sight
Software subcontract management
Software quality assurance
Software configuration management
The five CMM maturity levels
Level
1. Initial
Description
Features
The software process is ad hoc, and
occasionally even chaotic. Few processes
are defined, and success depends on
individual effort and heroics
• Overcommitment si common
• During a crisis, planned procedures are abandoned and project teams revert
to coding and testing
• Success depends on having an exceptional manager and seasoned and
effective software team
• The software team process is effectively a black box to the user community.
Resources go in and software potentially comes out
2. Repeatable Basic project management processes are
established to track cost, schedule, and
functionality. The necessary process
discipline is in place to repeat earlier
successes on projects with similar
applications
3. Defined
4. Managed
5. Optimized
Capability
Maturity
Model
• The planning and management of new projects is based on experience with similar projects.
• Process capability is enhanced at the project level by establishing basic process management
techniques
• Software requirements and deliverables are baselined
• Processes often differ between projects, reducing opportunities for teamwork and reuse
• The user community is provided visibility into the project at defined occasions, typically via the
review and acceptance of major project deliverables, allowing limited management control
1. Initial
Na této úrovni dominují nahodilé (ad hoc) procesy. Software
je vytvářen bez firemních pravidel chaoticky, takže se jeho
The software process
for both
management and
tvorba
často
dostává
do kritických
situací.
Dosažený
úspěch
• A standard process
is used, with
possible tailoring,
on all projects.
development activities is documented,
• Management has good insight into technical progress on the project
standardized,
and integrated
into a standard
ve
vývoji
software
je důsledkem
šťastné
náhody
a závisí
na into the project and
• Defined processes
allow the
user community
greater visibility
software process for the entire organization. All
enable accurate and rapid status updates.
projects use an approved, tailored
version of the
individuálních
schopnostech
a znalostech programátorů.
organization’s standard software process for
developing and maintaining
software.
Dohodnuté
termíny
nejsou většinou dodržovány a ukončení
vývoje
je dosahováno nesmírným
heroickým úsilím na konci
Detailed measures, called metrics, of the
• Productivity and quality are measured for important software process activities
software process and
product quality
are
across
all projects. jsou sečteny po ukončení
projektu.
Celkové
náklady
na
projekt
collected. Both the software process and
• The user community can establish an accurate, quantitative understanding of the
products are quantitatively
understood
and
software process
capability of
organization/team
and the project
vývoje
a posuzují
se jako
důsledek
vývoje
ayour
nutných
výdajů.
O risk before
controlled
the project begins.
kvalitě software se sice hovoří, ale nijak se nezajišťuje. Každý
Continuous process improvement is enabled
• Innovations that exploit the best software engineering practices are identified and
spoléhá
na toho
druhého,
žeshared
kvalitu
zajistí.
by quantitative feedback
from the
software
throughout
the organization
process and from piloting innovative ideas
and technologies.
• The software process is improved by changing common causes of inefficiency
• Disciplined change is the norm, not the exception.
The five CMM maturity levels
Capability
Maturity
Model
Level
Description
Features
1. Initial
The software process is ad hoc, and
occasionally even chaotic. Few processes are
defined, and success depends on individual
effort and heroics
• Overcommitment si common
• During a crisis, planned procedures are abandoned and project teams revert to
coding and testing
• Success depends on having an exceptional manager and seasoned and effective
software team
• The software team process is effectively a black box to the user community.
Resources go in and software potentially comes out
2. Repeatable Basic project management processes are
established to track cost, schedule, and
functionality. The necessary process
discipline is in place to repeat earlier
successes on projects with similar
applications
3. Defined
4. Managed
5. Optimized
The software process for both management and
development activities is documented,
standardized, and integrated into a standard
software process for the entire organization. All
projects use an approved, tailored version of the
organization’s standard software process for
developing and maintaining software.
• The planning and management of new projects is based on experience with similar
projects.
• Process capability is enhanced at the project level by establishing basic process
management techniques
• Software requirements and deliverables are baselined
• Processes often differ between projects, reducing opportunities for teamwork and reuse
• The user community is provided visibility into the project at defined occasions, typically
via the review and acceptance of major project deliverables, allowing limited
management control
• A standard process is used, with possible tailoring, on all projects.
• Management has good insight into technical progress on the project
• Defined processes allow the user community greater visibility into the project and
enable accurate and rapid status updates.
2. Repeatable
Opakovaně se dosahuje dobrých výsledků. Firma využívá
základních
postupů
projektového
řízení,
ale zsoftware
projektu
na
Detailed measures,
called metrics, of the
• Productivity
and quality are measured
for important
process activities
software process and product quality are
across all projects.
seandpřenášejí
jen
některé
úspěšné
prvky
řízení.
collected. Both theprojekt
software process
• The user
community
can establish
an accurate,
quantitative
understanding of the
products are quantitatively understood and
software process capability of your organization/team and the project risk before
Nicméně intuitivně
zaběhané procesy a povědomí, že je
controlled
the project begins.
potřeba pracovat• Innovations
kvalitně,
vytvářejí dost stabilní
Continuous process improvement is enabled
that exploit the best software engineering practices are identified and
by quantitative feedback
from the software
shared throughout
the organization
prostředí
pro udržení
přijatelné
úrovně jakosti
process and from piloting innovative ideas
• The software process is improved by changing common causes of inefficiency
and technologies. softwarových produktů.
• Disciplined change is the norm, not the exception.
The five CMM maturity levels
3. Defined
Capability
Maturity
Software je vyvíjen podle předem stanoveného postupu, metodicky,
Model
Level
Features
plánovitě,
sDescription
využitím pokročilého projektového
řízení s cílem dosáhnout
1. Initial
Thepožadovaného
software process is ad hoc,
and
si common
vypracování
software
v• Overcommitment
čase, s rozpočtovanými
náklady a
occasionally even chaotic. Few processes are • During a crisis, planned procedures are abandoned and project teams revert to
disponibilními
zdroji.
se pravidelné
odchylek od
defined,
and successProvádí
depends on individual
coding andvyhodnocování
testing
effort and heroics
• Success depends on having an exceptional manager and seasoned and effective
plánu a přijímají
se opatření ke krácení
termínů
jednotlivých činností, aby
software
team
• The software team
process is effectively
a black box to the
software byl dodán včas. O kvalitu produktů
a služeb
se s ohledem
nauser community.
Resources go in and software potentially comes out
zákazníky
usiluje,
proto
je• kvalita
softwaru
dodržována
na with similar projects.
The planning and
management of new
projects is based on experience
2. Repeatable explicitně
Basic project management
processes
are
• Process capability is enhanced at the project level by establishing basic process management
to track cost, schedule, and
velmi dobréestablished
úrovni
a má tendenci vykazovat
techniques určité zlepšování.
functionality. The necessary process
discipline is in place to repeat earlier
successes on projects with similar
applications
• Software requirements and deliverables are baselined
• Processes often differ between projects, reducing opportunities for teamwork and reuse
• The user community is provided visibility into the project at defined occasions, typically via the
review and acceptance of major project deliverables, allowing limited management control
The software process for both management
and development activities is documented,
standardized, and integrated into a standard
software process for the entire organization.
All projects use an approved, tailored version
of the organization’s standard software
process for developing and maintaining
software.
• A standard process is used, with possible tailoring, on all projects.
• Management has good insight into technical progress on the project
• Defined processes allow the user community greater visibility into the
project and enable accurate and rapid status updates.
4. Managed
Detailed measures, called metrics, of the
software process and product quality are
collected. Both the software process and
products are quantitatively understood and
controlled
• Productivity and quality are measured for important software process activities
across all projects.
• The user community can establish an accurate, quantitative understanding of the
software process capability of your organization/team and the project risk before
the project begins.
5. Optimized
Continuous process improvement is enabled
by quantitative feedback from the software
process and from piloting innovative ideas
and technologies.
• Innovations that exploit the best software engineering practices are identified and
shared throughout the organization
• The software process is improved by changing common causes of inefficiency
• Disciplined change is the norm, not the exception.
3. Defined
The five CMM maturity levels
Capability
Maturity
Model
4. Manage
1. Initial
The software process is ad hoc, and
• Overcommitment si common
Firma
má Few
všechny
procesy
definovány
a stanoven
occasionally
even chaotic.
processes are
• During ajasně
crisis, planned
procedures are abandoned
and projectpro
teams revert to
defined, and success depends on individual
coding and testing
postup měření, který
vyhodnocuje
podíl
a and seasoned and effective
effort andně
heroics
• Success
depends on having jejich
an exceptional
manager
software team
efektivitu při tvorbě •software.
Zjištěné charakteristiky
The software team process is effectively a black box to the user community.
go in and software
comes
out
procesů jsou postupně Resources
upravovány
tak,potentially
aby se
firma
• The planning and management of new projects is based on experience with similar projects.
2. Repeatable Basic project management processes are
přizpůsobila
měnícím
se
podmínkám
trhu,
aniž
to mělo
• Process
capability is enhanced
at the project
levelby
by establishing
basic process management
established
to track cost, schedule,
and
techniques
functionality. The necessary process
dopad na jakost vyvíjeného
softwaru,
jehož
kvalitativní
• Software requirements
and deliverables
are baselined
discipline is in place to repeat earlier
• Processes often differ between projects, reducing opportunities for teamwork and reuse
successes
on projects with similar
parametry
jsou firmou
cílevědomě
stále
zvyšovány
a occasions, typically via the
• The
user community is provided
visibility
into the project at defined
applications
review and acceptance of major project deliverables, allowing limited management control
dosahují vysoké úrovně.
Level
Description
3. Defined
The software process for both management and
development activities is documented,
standardized, and integrated into a standard
software process for the entire organization. All
projects use an approved, tailored version of the
organization’s standard software process for
developing and maintaining software.
• A standard process is used, with possible tailoring, on all projects.
• Management has good insight into technical progress on the project
• Defined processes allow the user community greater visibility into the project and
enable accurate and rapid status updates.
Detailed measures, called metrics, of the
software process and product quality are
collected. Both the software process and
products are quantitatively understood
and controlled
• Productivity and quality are measured for important software process
activities across all projects.
• The user community can establish an accurate, quantitative understanding
of the software process capability of your organization/team and the project
risk before the project begins.
Continuous process improvement is enabled
by quantitative feedback from the software
process and from piloting innovative ideas
and technologies.
• Innovations that exploit the best software engineering practices are identified and
shared throughout the organization
• The software process is improved by changing common causes of inefficiency
• Disciplined change is the norm, not the exception.
4. Managed
5. Optimized
Features
The five CMM maturity levels
Capability
Maturity
Model
Level
Description
Features
1. Initial
The software process is ad hoc, and
occasionally even chaotic. Few processes are
defined, and success depends on individual
effort and heroics
• Overcommitment si common
• During a crisis, planned procedures are abandoned and project teams revert to
coding and testing
• Success depends on having an exceptional manager and seasoned and effective
software team
• The software team process is effectively a black box to the user community.
Resources go in and software potentially comes out
5. Optimizing
Neustálá zpětná vazba ovlivňuje následné softwarové
• The planning and management of new projects is based on experience with similar projects.
2. Repeatable Basic project management processes are
projekty tak, aby se firemní
procesy
neustále
• Process capability
is enhanced
at the project level by establishing basic process management
established to track cost, schedule, and
techniques
functionality.
The
necessary
process
zlepšovaly a dosahovaly • předem
definovaných,
Software requirements
and deliverables are baselined
discipline is in place to repeat earlier
• Processes often differ between projects, reducing opportunities for teamwork and reuse
successes
on projects with similar
optimálních
parametrů. •Firma
dosahuje
špičkové
The user community
is providedtrvale
visibility into the
project at defined occasions, typically via the
applications
review and acceptance of major project deliverables, allowing limited management control
jakosti, aniž by náklady na kvalitu softwaru měly dopad
The software
process for both management
and
3. Defined
• A standard garantovaná
process is used, with possible
tailoring, on all projects.
na hospodaření
firmy.
Naopak,
kvalita
development activities is documented,
• Management has good insight into technical progress on the project
standardized,
and integrated
into a standard
software
usnadňuje
jeho• Defined
prodej.
processes allow the user community greater visibility into the project and
software process for the entire organization. All
projects use an approved, tailored version of the
organization’s standard software process for
developing and maintaining software.
4. Managed
Detailed measures, called metrics, of the
software process and product quality are
collected. Both the software process and
products are quantitatively understood and
controlled
5. Optimized
Continuous process improvement is
enabled by quantitative feedback from
the software process and from piloting
innovative ideas and technologies.
enable accurate and rapid status updates.
• Productivity and quality are measured for important software process activities
across all projects.
• The user community can establish an accurate, quantitative understanding of the
software process capability of your organization/team and the project risk before
the project begins.
• Innovations that exploit the best software engineering practices are
identified and shared throughout the organization
• The software process is improved by changing common causes of
inefficiency
• Disciplined change is the norm, not the exception.
CMM
•
Model se poměrně rychle v průběhu osmdesátých let rozšířil a řada
dalších univerzit a firem rozpracovávala a propracovávala jeho dílčí
aspekty. Pro model byly definovány jednotlivé kroky, které
umožňují postup na vyšší úroveň:
•
•
z 1. na 2. úroveň - disciplína, zodpovědnost, pořádek, cílevědomost,
z 2. na 3. úroveň - standardizace, stálost a provázanost firemních
procesů, monitorování procesů,
ze 3. na 4. úroveň - kritéria a měření procesů, řízení změn a
využívání predikce vývoje procesů,
ze 4. na 5. úroveň - neustálé zlepšování procesů, vícekriteriální
optimalizace procesů, adaptibilita procesů.
•
•
CMM
• Rovněž pro jednotlivé úrovně byly
definovány klíčové aspekty úspěchu
(key process areas).
The method used for estimation
COCOMO Model
The original COCOMO model was first published by Barry Boehm in 1981 at CSE Center for Software Engineering. This is an acronym
derived from the first two letters of each word in the longer phrase Constructive Cost Model.
The COCOMO cost estimation model is used by thousands of software project managers, and is based on a study of hundreds of
software projects. The most fundamental calculation in the COCOMO model is the use of the Effort Equation to estimate the number of
man-months required to develop a project. Most of the other COCOMO results, including the estimates for Requirements and
Maintenance, are derived from this quantity. The original COCOMO model has been very successful, but it doesn't apply to newer
software development practices as well as it does to traditional practices, but there is COCOMO II tuned to modern software life cycles.
COCOMO II targets the software projects of the 1990s and 2000s.
The COCOMO calculations are in the form of non-linear mathematical functions and are based on estimates of a project's size in Source
Lines of Code (SLOC). Then the COCOMO model makes its estimates of required effort (measured in man-months) based on the
estimate of the software project's size. From this value, the prediction of following values can be derived:
• project duration (typically in months) required to complete the software project, and
• required staffing (typically in FTE).
COCOMO is a model that allows one to estimate the cost, effort, and schedule when planning a new software development
activity.
Function Points / Feature Points
The Function Point methodology was developed in 1979 by Allan Albrecht at IBM and was modified in 90’s as the Feature Point
methodology by Jones. This methodology is based on the premise that the size of a software project can be estimated early, during the
requirements analysis. If there are counted some requirement characteristics of the project, and then multiplied by weighting and adjustment
factors, the sum is the total point count.
Units counted through either Function Points or Feature Points represent the expected complexity and required effort of the
project.
Our approach
Since all data were not available in the form required as inputs for the COCOMO method and only a basic information about
Function Points and related project sizes in man-months were on hand, the man-months obtained from known Function Points
were converted into its comparable COCOMO equivalent - the Effort value. This value was then used in the COCOMO
equations to make all subsequent estimates.
Complexity of applications in function points
Function Points (Albrecht, 1979) is a metrics used for estimating the size of an application.
The basic idea of Function Points is that five items are counted:
1) the inputs to the application
2) the outputs from it
3) inquiries by users
4) the data files that would be updated by the application, and
5) the interface to the application.
Once these figures are known, they are multiplied by following coefficients: 4, 5, 4, 10, and 7 (these values can be modified from own
experience, if required). The result indicates the estimated size of an application. Function Points worked reasonably for simple
transaction processing or management information systems in 80’s, but unfortunately they often proved to be inadequate for estimating
complex application such as information systems for high-tech industries.
Therefore, there exists the need for Feature Points (Jones, 1995), a superset of classical FP, to correct this problem. Feature Points
introduces new measurement, algorithms, giving it a weighting of 3. Additionally, the data file weighting is decreased from 10 to 7.
Productivity rates
in FP per man/month
> 100
75 - 100
50 - 75
25 - 50
15 - 25
5 - 15
1-5
<1
Software Project Size
100 FP
1000 FP
10000 FP
1.0%
3.0%
7.0%
15.0%
40.0%
25.0%
10.0%
4.0%
0.0%
0.0%
0.0%
0.1%
1.4%
13.5%
70.0%
15.0%
0.01%
0.1 %
1.0%
5.0%
10.0%
50.0%
30.0%
4.0%
Programming Language
(Jones)
Assembler
low level: C
classical: Fortran, Cobol
structured: Pascal, RPG, PL1
hybrid object oriented: Modula-2, C++
non imperative: Lisp, Prolog
4GL languages
obj. oriented: VB, Java, C#
pure object oriented: Smalltalk, CLOS
query languages: SQL, QBE, OQL
Source code statements
required for one FP
320
150
106
80
71
64
40
32
21
16
(Jones)
Metrics management is the process of collecting,
summarizing and acting on measurements
Metrics management is used to potentially improve both the quality of the software that is developed or maintained and also to improve the
actual software processes of the organization itself. Metrics management provides the information critical to understanding the quality of
the work that the project team has performed, and how effectively that work was performed.
Software companies are used to collect metrics because this is the only proven tool to obtain managerial information for software
development and quality assurance In the suggested Capability Maturity Model (CMM), there are recommended numbers of metrics to be
collected and analyzed. Optimal frequency of collect the recommended metrics is weekly (or earlier, if there is a tool), and optimal
frequency of analyze metrics is between weekly and monthly.
Why to Collect Metrics?
 To support greater predictability and accuracy
of schedules and estimates.
 To support quality assurance efforts by
identifying the techniques that work best and
the areas that need more work.
 To support productivity and process
improvements efforts by measuring how
efficient the IT staff is and by indicating
where the IT staff need improve.
 To improve management control by tracking
both projects and people.
 To improve the motivation of developers by
making them aware of what works and what
does not.
 To improve communications by describing
accurately what is happening, and by
identifying trends early on.
The need for Metrics in CMM
CMM Level Number
Recommended Number of Metrics
1
0 - 11
2
11 - 22
3
22 - 30
4
30 - 36
5
36 - 50
Family of potential SW metrics
Stage
Potential Metrics
Define Requirements and Justify
Number of use case
Function/feature points
Level of risk
Project size
Define Management Documents
Cost/benefit breakpoint
Number of reused infrastructure artifacts
Number of introduced infrastructure artifacts
Requirements instability
Model
Procedure/Operation count of a module
Number of variables per data structure
Size of procedures/operations
Number of data types per database
Number of data relationships per database
Number of interfaces per module
Requirements instability
Program, Generalize and Test
Procedures/Operations size
Procedures/Operations response
Comments per procedure/operation
Percentage of commented procedures/operations
Global usage
Number of candidate items for generalization
Percentage of items generalized
Effort required to generalize an item
Percentage of deliverables reviewed
Time to fix defects
Defect recurrence
Defect type recurrence
Recommended metrics are printed in bold face.
Family of potential SW metrics
Stage
Potential Metrics
Deliver and Test
Time to fix defects
Defect recurrence
Defect type recurrence
Number of defects
Defect source count
Work effort to fix a defect
Percentage of items reworked
Enhancements implemented per release
Amount of documentation
Percentage of customers trained
Average training time per user trainer
Assess
Number of lessons learned
Percentage of staff members assessed
Support
Average response time
Average resolution time
Support request volume
Support backlog
Support request aging
Support engineer efficiency
Reopened support requests
Mean time between failures
Software change requests opened and closed
Recommended metrics are printed in bold face.
Metrics Applicable to All of the Phases and Stages
Metrics
Description
Calendar time expended This schedule metric is a measure of a calendar time to complete a task, where a task can be as small as
the creation of a deliverable or as large as a project phase or an entire project.
Overtime
This cost metric is a measure of the amount of overtime for a project and can be collected as either an amount of of
workdays or a percentage of overall effort. This metric is often subdivided by overtime for work directly related to
development, such as programming or testing, and by indirect effort, such as training or team coordination.
Staff turnover
This productivity metric is a measure of the number of people gained and lost over a defined period of
time, typically a calendar month. This metric is often collected by position/role and can be used to help
define human resources requirements and the growth rate of your information technology (IT) department.
Work effort expended
This cost metric is a measure of the amount of work expended to complete a task, typically measured in
work days or work months. This metric is often collected by task (see above).
Metric categories
Category
Description
Cost
Cost metrics measure the amount of money invested in a project. Examples of cost metrics include effort expended
or investment in training.
Productivity
Productivity metrics measure the effectiveness of the organization`s infrastructure. Examples of productivity metrics
include the number of reused infrastructure items and trend analysis of effort expended over given periods of time.
Quality
Quality metrics measure the effectiveness of the quality assurance efforts. Examples of quality assurance metrics
include the percentage of deliverables reviewed and defect type recurrence.
Requirements
Requirements metrics measure the effectiveness of the requirements definition and validation efforts. Examples of
requirements metrics include requirements instability and number of use cases.
Schedule
Schedule metrics measure the accuracy of your proposed schedule to the actual schedule. Examples of schedule
metrics include the calendar time expended to perform a task or project phase.
Size
Size metrics measure, as the name suggests, the size of the development efforts. Examples of size metrics include
the number of methods of a class and the function/feature point count.
Testing
Testing metrics measure the effectiveness of the testing efforts. Examples of testing metrics include the defect
severity count and the defect source count.
Závěr
• Bez metrik nelze provádět kvalitně
řízený softwarový proces nemluvě o
jeho optimalizaci.
• CMM nám ukazuje, jak na tom jsme s
úrovní řízení softwarového procesu!
Vývoj rozsáhlých ICT produktů
si žádá vědomé řízení celého
projektového procesu!
Descargar

Slide 1