Quality of software
(development)
Krzysztof Bronk, Łukasz Jopek
Krzysztof Bronk, Łukasz Jopek
Quality of software
(development)
Krzysztof Bronk, Łukasz Jopek
Krzysztof Bronk, Łukasz Jopek
For your information
For your information
This presentation is organized in
This presentation is organized in
detailed points, so that all the
detailed points, so that all the
necessary information is available
necessary information is available
Straight-forward explanation
Straight-forward explanation
What is software
What is software
(development) quality?
(development) quality?
all the actions leading to
all the actions leading to
satisfaction of all the interested
satisfaction of all the interested
people in the project
people in the project
assurance of the project meeting
assurance of the project meeting
the requirements and standards
the requirements and standards
Why is it difficult to
Why is it difficult to
measure quality?
measure quality?
Many of the criteria are intangible
Many of the criteria are intangible
Those happen to be most important
Those happen to be most important
Measures should be accurate
Measures should be accurate
Complexity of the final product
Complexity of the final product
Test data explosion
Test data explosion
Multifaceted approach (wieloaspektowe podejście)
Multifaceted approach (wieloaspektowe podejście)
Long term use
Long term use
Criteria have to be known before project creation
Criteria have to be known before project creation
Different use
Different use
Scalability
Scalability
Time, money costs
Time, money costs
Speculation and assumptions, patterns, heuristics
Speculation and assumptions, patterns, heuristics
TQM
TQM
Total Quality Management
Total Quality Management
Author: Eiji Toyoda
Author: Eiji Toyoda
Aim: motorization industry repair in 1950
Aim: motorization industry repair in 1950
Formalized by Americans (Deming, Crosby, Juran,
Formalized by Americans (Deming, Crosby, Juran,
Feigenbaum) and Japanese (Toyoda, Imai, Ishikawa)
Feigenbaum) and Japanese (Toyoda, Imai, Ishikawa)
and British (Oakland).
and British (Oakland).
ISO definition:
ISO definition:
"TQM is a management approach for an
"TQM is a management approach for an
organization, centred on quality, based on the
organization, centred on quality, based on the
participation of all its members and aiming at long-
participation of all its members and aiming at long-
term success through customer satisfaction, and
term success through customer satisfaction, and
benefits to all members of the organization and to
benefits to all members of the organization and to
society."
society."
My definition: “Client is happy
My definition: “Client is happy
We are happy”
We are happy”
Quality in ISO 9000
Quality in ISO 9000
ISO 9000 - a family of ISO (the International
ISO 9000 - a family of ISO (the International
Organization for Standardization) standards for quality
Organization for Standardization) standards for quality
management systems
management systems
Primarily concerns with "quality management", which
Primarily concerns with "quality management", which
means what the company does to fulfil:
means what the company does to fulfil:
–
the customer's quality requirements
the customer's quality requirements
–
applicable regulatory requirements, while aiming to
applicable regulatory requirements, while aiming to
–
enhance customer satisfaction
enhance customer satisfaction
–
achieve consecutive performance improvements
achieve consecutive performance improvements
ISO 9000 terminology
ISO 9000 terminology
A
A
quality
quality
is a desirable characteristic, customers
is a desirable characteristic, customers
expectations, that a product or service must have
expectations, that a product or service must have
Quality system
Quality system
is a organization pattern of
is a organization pattern of
responsibilities, authorities, and relationships that
responsibilities, authorities, and relationships that
control how people perform their functions
control how people perform their functions
Quality management
Quality management
includes all the activities that
includes all the activities that
managers carry out in an effort to implement their
managers carry out in an effort to implement their
quality policy
quality policy
A
A
quality policy
quality policy
statement defines or describes an
statement defines or describes an
organization's commitment to quality
organization's commitment to quality
Quality audits
Quality audits
examine the elements of a quality
examine the elements of a quality
management system in order to evaluate how well
management system in order to evaluate how well
these elements comply with quality system
these elements comply with quality system
requirements
requirements
Policy & system quality
Policy & system quality
(1)
(1)
Quality policy should:
Quality policy should:
–
State a clear commitment to quality
State a clear commitment to quality
–
Recognize customer needs and expectations
Recognize customer needs and expectations
–
Be actively supported by senior management
Be actively supported by senior management
–
List the quality objectives you want to achieve
List the quality objectives you want to achieve
–
Be understood by everyone in the organization
Be understood by everyone in the organization
–
Be consistent with your organization's goal
Be consistent with your organization's goal
–
Be applied and maintained throughout your
Be applied and maintained throughout your
organization
organization
Policy & system quality
Policy & system quality
(2)
(2)
Quality system should:
Quality system should:
–
Describe the structure of your organization
Describe the structure of your organization
–
Discuss your quality system procedures
Discuss your quality system procedures
–
Define quality documents and records
Define quality documents and records
–
Set responsibilities and objectives
Set responsibilities and objectives
Software quality
Software quality
planning
planning
ISO 9000
ISO 9000
–
Has a "manufacturing" focus and was not written
Has a "manufacturing" focus and was not written
with software in mind
with software in mind
–
Designed to be a generic standard, applicable to any
Designed to be a generic standard, applicable to any
business
business
ISO 9000-3
ISO 9000-3
–
Designed to assist software development
Designed to assist software development
–
Deals with potential complexity and interaction
Deals with potential complexity and interaction
among software subsystems and software-hardware
among software subsystems and software-hardware
subsystems, and a unique product life cycle
subsystems, and a unique product life cycle
IEEE 610-12
IEEE 610-12
–
Quality measurement is based on the combination of
Quality measurement is based on the combination of
desirable features
desirable features
ISO 9126
ISO 9126
Six groups that build up quality
Six groups that build up quality
Functionality
Functionality
Reliability
Reliability
Usability
Usability
Effectiveness
Effectiveness
Maint
Maint
enance
enance
Portability
Portability
Functionality
Functionality
Meeting requirements
Meeting requirements
Careful examination
Careful examination
Precisely designed
Precisely designed
Precise
Precise
Liaison (współdziałanie)
Liaison (współdziałanie)
Security
Security
Reliability
Reliability
Mature product
Mature product
Error tolerance
Error tolerance
System restore after crash
System restore after crash
Usability
Usability
Easy to understand
Easy to understand
Easy to use
Easy to use
Effectiveness
Effectiveness
Reasonable response time
Reasonable response time
Reasonable resource use
Reasonable resource use
Maint
Maint
enance
enance
Easy to change
Easy to change
Easy to validate
Easy to validate
Easy to modify
Easy to modify
Stable
Stable
Available
Available
Portability
Portability
Adjustability
Adjustability
Installation in different
Installation in different
environment
environment
Functional correspondence
Functional correspondence
(zgodność)
(zgodność)
Quality Management
Quality Management
Client oriented
Client oriented
Good leadership
Good leadership
Staff satisfaction
Staff satisfaction
Staff motivation
Staff motivation
Staff training
Staff training
Process approach: processes as steps, measures of
Process approach: processes as steps, measures of
processes, relationship between processes
processes, relationship between processes
System approach: detailed control of the whole
System approach: detailed control of the whole
developing environment
developing environment
Good management = combination of both
Good management = combination of both
Evolution, not revolution
Evolution, not revolution
Feedback
Feedback
Close relationship with client for the sake of the project
Close relationship with client for the sake of the project
Software quality
Software quality
assurance
assurance
Software Quality Assurance
Software Quality Assurance
(SQA) is defined as “a
(SQA) is defined as “a
planned and systematic approach to the evaluation of
planned and systematic approach to the evaluation of
the quality of and adherence to software product
the quality of and adherence to software product
standards, processes, and procedures”
standards, processes, and procedures”
Software QA involves the entire software development
Software QA involves the entire software development
process - monitoring and improving the it, making sure
process - monitoring and improving the it, making sure
that any agreed-upon standards and procedures are
that any agreed-upon standards and procedures are
followed
followed
In most cases, because of the software complexity, we
In most cases, because of the software complexity, we
inspect crucial parts only
inspect crucial parts only
Software Quality Assurance Plan
Software Quality Assurance Plan
(SQAP) defines
(SQAP) defines
inspection procedures in each of the projects phases
inspection procedures in each of the projects phases
Quality loss risk
Quality loss risk
If the project is new
If the project is new
If the project is complex
If the project is complex
Staff not trained well
Staff not trained well
Inexperienced staff
Inexperienced staff
New procedures
New procedures
Low organizational maturity of the
Low organizational maturity of the
developer
developer
How to lower the risk
How to lower the risk
SQA should start the work early
SQA should start the work early
Check the user requirements,
Check the user requirements,
plans, standards
plans, standards
Higher cost of an error if detected
Higher cost of an error if detected
too late
too late
Quality assurance
Quality assurance
methods
methods
(SQM)
(SQM)
Tasks from organization perspective:
Tasks from organization perspective:
–
Developing process maintenance, supervision, and
Developing process maintenance, supervision, and
acknowledgement
acknowledgement
–
Standards usage
Standards usage
Tasks from project perspective:
Tasks from project perspective:
–
Adapt standards
Adapt standards
–
Project reviews
Project reviews
–
Testing
Testing
–
Taking part in inspections
Taking part in inspections
–
Developing and qualities plans evaluation
Developing and qualities plans evaluation
–
System configuration maintenance audits
System configuration maintenance audits
–
Participation in managing committee
Participation in managing committee
Quality assurance
Quality assurance
methods
methods
(SQM)
(SQM)
Staff should verify if:
Staff should verify if:
–
Project is well organized
Project is well organized
–
Projects life cycle is appropriate
Projects life cycle is appropriate
–
Team members have assigned tasks and
Team members have assigned tasks and
responsibilities
responsibilities
Staff should also:
Staff should also:
–
Supervise documentation
Supervise documentation
–
Check measurements data
Check measurements data
–
Plan controls, audits, tests
Plan controls, audits, tests
–
Register problems and solutions
Register problems and solutions
–
Approve development tools, techniques, methods
Approve development tools, techniques, methods
–
Approve software storage
Approve software storage
Main quality assurance
Main quality assurance
methods groups
methods groups
Systems certification before the implementation
Systems certification before the implementation
Enforcing standards of gathering and processing data
Enforcing standards of gathering and processing data
Reviewing and certification of developing and
Reviewing and certification of developing and
documentation
documentation
System architecture standards and programming
System architecture standards and programming
patterns developing
patterns developing
Project reviewing in sense of compatibility
Project reviewing in sense of compatibility
New and modified software testing
New and modified software testing
Developing managing standards
Developing managing standards
Trainings
Trainings
Quality norms – IEEE-
Quality norms – IEEE-
730
730
General quality assurance plan
General quality assurance plan
Analysis of the point of view
Analysis of the point of view
Reference of the developer
Reference of the developer
Software management
Software management
Documentation
Documentation
Standardization of actions
Standardization of actions
Inspections
Inspections
Audits
Audits
Software configuration management
Software configuration management
Reports of problems
Reports of problems
Used methods and tools
Used methods and tools
Code control
Code control
Data warehouse management
Data warehouse management
Code maintenance
Code maintenance
IEEE-983 as more detailed IEEE-730
IEEE-983 as more detailed IEEE-730
Developing processes
Developing processes
maturity and
maturity and
immaturity
immaturity
Developing processes immaturity factors:
Developing processes immaturity factors:
–
Improvisation during the developing process
Improvisation during the developing process
–
Process is specified, but the specification is not used
Process is specified, but the specification is not used
–
Immediate crisis actions
Immediate crisis actions
–
Timetable and budget are being exceeded
Timetable and budget are being exceeded
–
Constantly decreased functionality
Constantly decreased functionality
–
Low-grade product quality
Low-grade product quality
–
No objective estimation criteria
No objective estimation criteria
Developing processes
Developing processes
maturity and
maturity and
immaturity
immaturity
Developing processes maturity factors:
Developing processes maturity factors:
–
Software building ability is a characteristic of an
Software building ability is a characteristic of an
organization, and not a person
organization, and not a person
–
Process is determined, known and used
Process is determined, known and used
–
Process is analysed and improved
Process is analysed and improved
–
Tasks are planned and monitored
Tasks are planned and monitored
–
Roles and responsibilities are defined
Roles and responsibilities are defined
–
Objective and quantitative estimation is possible
Objective and quantitative estimation is possible
Capability Maturity
Capability Maturity
Model
Model
(CMM)
(CMM)
Initiated in response to the request of the U.S.
Initiated in response to the request of the U.S.
Government to provide a method for assessing the
Government to provide a method for assessing the
capability of its contractors
capability of its contractors
The CMM is a framework that describes the key
The CMM is a framework that describes the key
elements of an effective process
elements of an effective process
Describes an evolutionary improvement path from an
Describes an evolutionary improvement path from an
ad hoc, immature process to a mature, disciplined
ad hoc, immature process to a mature, disciplined
process
process
The CMM covers practices for planning, engineering,
The CMM covers practices for planning, engineering,
and managing development and maintenance activities
and managing development and maintenance activities
Partially based on TQM concept
Partially based on TQM concept
CMM's Five Maturity
CMM's Five Maturity
Levels of Software
Levels of Software
Processes
Processes
CMM's Five Maturity
CMM's Five Maturity
Levels of Software
Levels of Software
Processes
Processes
At the
At the
initial level
initial level
, processes are disorganized, even chaotic. Success
, processes are disorganized, even chaotic. Success
is likely to depend on individual efforts, and is not considered to be
is likely to depend on individual efforts, and is not considered to be
repeatable, because processes would not be sufficiently defined and
repeatable, because processes would not be sufficiently defined and
documented to allow them to be replicated.
documented to allow them to be replicated.
At the
At the
repeatable level
repeatable level
, basic project management techniques are
, basic project management techniques are
established, and successes could be repeated, because the requisite
established, and successes could be repeated, because the requisite
processes would have been made established, defined, and
processes would have been made established, defined, and
documented.
documented.
At the
At the
defined level
defined level
, an organization has developed its own standard
, an organization has developed its own standard
software process through greater attention to documentation,
software process through greater attention to documentation,
standardization, and integration.
standardization, and integration.
At the
At the
managed level
managed level
, an organization monitors and controls its own
, an organization monitors and controls its own
processes through data collection and analysis.
processes through data collection and analysis.
At the
At the
optimizing level
optimizing level
, processes are constantly being improved
, processes are constantly being improved
through monitoring feedback from current processes and introducing
through monitoring feedback from current processes and introducing
innovative processes to better serve the organization's particular
innovative processes to better serve the organization's particular
needs.
needs.
Software quality
Software quality
assurance plan
assurance plan
Maintained through the whole
Maintained through the whole
development life cycle
development life cycle
First version a
First version a
t the end of the
t the end of the
user
user
requirements phase
requirements phase
Size should resemble the size of
Size should resemble the size of
the project
the project
Should deal with all recognized
Should deal with all recognized
aspects of quality assurance
aspects of quality assurance
SQAP sections
SQAP sections
SQAP for user requirements
SQAP for user requirements
SQAP for analysis
SQAP for analysis
SQAP for project architecture
SQAP for project architecture
SQAP for design phase
SQAP for design phase
SQAP for implementation and
SQAP for implementation and
deployment
deployment
Basic rules
Basic rules
Numbers for points should not be
Numbers for points should not be
changed
changed
Additional information should not
Additional information should not
be included in the main document
be included in the main document
part
part
Document easy to navigate
Document easy to navigate
Easy to understand
Easy to understand
Coherent (spójny)
Coherent (spójny)
Modifiable
Modifiable
What’s inside - Aim
What’s inside - Aim
SQAP aim
SQAP aim
Type of reader
Type of reader
Software products related to
Software products related to
SQAP
SQAP
Wilful use of software (name the
Wilful use of software (name the
development phase)
development phase)
What’s inside -
What’s inside -
Management
Management
Organizations dealing with quality
Organizations dealing with quality
Roles (with no assignments)
Roles (with no assignments)
Organization’s responsibility
Organization’s responsibility
What’s inside –
What’s inside –
Documentation
Documentation
Identifies all documents produced
Identifies all documents produced
in this phase
in this phase
How to check those documents
How to check those documents
for user requirements conformity
for user requirements conformity
(zgodność)
(zgodność)
What’s inside –
What’s inside –
Standards
Standards
Details on standards and
Details on standards and
references to other documents
references to other documents
What’s inside –
What’s inside –
Inspections/Audits
Inspections/Audits
Describes methods of monitoring
Describes methods of monitoring
of plan and procedure
of plan and procedure
correspondence
correspondence
Role of SQA personnel in those
Role of SQA personnel in those
procedures
procedures
What’s inside – Tests
What’s inside – Tests
How to monitor and verify the
How to monitor and verify the
verification process and how to
verification process and how to
check the acceptation tests
check the acceptation tests
What’s inside –
What’s inside –
Problem reports
Problem reports
How to signify an error of problem
How to signify an error of problem
How to measure errors
How to measure errors
Error reaction procedure
Error reaction procedure
What’s inside – Code
What’s inside – Code
control
control
How to maintain clear code, make
How to maintain clear code, make
it portable, how to store it
it portable, how to store it
securely
securely
Code documentation procedures
Code documentation procedures
What’s inside – Media
What’s inside – Media
control
control
Procedures regarding media, on
Procedures regarding media, on
which the software and
which the software and
documentation will be stored
documentation will be stored
What’s inside –
What’s inside –
Provider
Provider
Know your provider
Know your provider
control procedures
control procedures
What’s inside – Notes
What’s inside – Notes
maint
maint
enance
enance
procedures regarding notes from
procedures regarding notes from
meetings or general information
meetings or general information
exchange and how to store them
exchange and how to store them
properly
properly
who has access to notes
who has access to notes
Apart from google &
Apart from google &
Włodek
Włodek
http://www.mhhe.com/
http://www.mhhe.com/