Filetype PDF | Posted on 30 Jan 2023 | 3 years ago
Authentication
digital resources
302x Filetype PDF File size 0.06 MB Source: www.iso.org
Proc. 2013 Canadian Engineering Education Association (CEEA13) Conf.
Software Quality Assurance
in an Undergraduate Software Engineering Program
Claude Y. Laporte, Alain April
École de technologie supérieure (ÉTS)
claude.laporte@etsmtl.ca, alain.april@etsmtl.ca
Abstract – Software tests are used by most The École de technologie supérieure (ÉTS) began
organizations. However, many other software quality offering its undergraduate software engineering program
assurance practices are often neglected. Most developers in 2001. Recent publications have described the extensive
are not aware of the high cost of inferior quality and its use of capstone projects, tools, and frameworks in that
impact on the duration and budget of a project. At the program [8, 9]. The aim of this specific SQA course (see
École de technologie supérieure (ÉTS), software quality LOG330 in Table 1), which is mandatory in the ÉTS
assurance (SQA) is taught in lecture format in the software engineering curriculum, is to ensure that
undergraduate software engineering curriculum. The software engineering students are aware of the importance
SQA course covers the concepts of the business model of SQA, and that they understand and are able to manage
and the cost of quality, to convince students of the and apply SQA practices in real situations. This also
importance of putting in place adequate prevention and includes hands-on knowledge of the key ISO and IEEE
evaluation practices, both to reduce the number of defects standards, as well as how to use industrial and open
and to predict the extra effort needed to correct defects source SQA tools in practice. The course allows students
introduced as the work progresses. to apply a wide range of SQA practices throughout a
software development cycle in a capstone project.
The course includes a 10-week capstone project in which
teams of 4 students apply the SQA practices taught in The professors who designed the SQA course, and are
class in a software development assignment. The students now teaching it, have combined industrial experience of
collect measures throughout the 10-week period, and the more than 20 years, mainly in the telecommunication,
performance of each team is analyzed. This analysis defense, and railway sectors. The course is made up of
allows discussion to take place on the positive impact of lectures, practical exercises, and a team project. A
SQA practices as a way to deliver quality software on continuous process of student evaluation is carried out to
time and within budget. ensure that the concepts are well understood. The
assessments are performed using exams, laboratory
Keywords: software quality assurance, standards, sessions, and mini-tests. The software tools, mostly open
ISO/IEC 29110, cost of quality, capstone project. source, provide the necessary support to students to
enable them to work with SQA as it is performed in
industry.
1. INTRODUCTION
This article is divided into two sections. First, the
As reported by Charette [2], software specialists spend authors present an overview of the undergraduate
about 40 to 50 percent of their time on avoidable rework. program in software engineering offered at the ÉTS. They
The ability of organizations to compete, adapt, and then present a detailed description of the SQA course, and
survive is increasingly dependent on quality, productivity, of the laboratory sessions.
development time, and cost. Systems and software are
growing larger and more complex every year. For 2. OVERVIEW OF THE SOFTWARE
example, top-of-the-line cars contain up to 100 million ENGINEERING CURRICULUM
lines of code, 80 processors, and 5 bus systems (Charette,
2005). Software quality assurance (SQA) becomes even The education system in Quebec is somewhat different
more important when we consider all the software from that in the rest of North America. Students attend
development projects that have failed, and the financial secondary school for only five years, followed by two or
losses generated by those failures. three years at a Cégep, or General and Vocational
College, which offers college-level programs for entry
into university (with two-year pre-university programs) or
CEEA13; Paper 109
Montreal, QC; June 17-20, 2013 – 1 of 6 –
Proc. 2013 Canadian Engineering Education Association (CEEA13) Conf.
training for students for technical or technological careers, the curriculum, and excludes courses such as
the latter being roughly the equivalent of community mathematics, physics, management, and the social
colleges elsewhere in North America. A student planning sciences, which are common to all undergraduate
to attend university would spend two years at a Cégep, engineering programs at the school.
followed by three or four years at a university to earn an
undergraduate degree. However, the ÉTS is exceptional, Table 1: List of software engineering courses at the ÉTS
in that its programs are designed for students who have Course ID Course title
completed a three-year technology-oriented Cégep
program. As a consequence, all students accepted at the LOG121 Object Oriented
ÉTS have received training in a technical discipline LOG210 Software Analysis and Design
related to engineering. In the case of software engineering LOG240 Tests and Maintenance
(SE), this means that the students have three years of LOG320 Data Structures and Algorithms
programming education and experience prior to their LOG330 Software Quality Assurance
arrival at the ÉTS. We are therefore able to teach LOG350 User Interface Design and Evaluation
concepts, such as OOD, OOA, and design patterns, during LOG410 Needs and Requirements Analysis
their first year, and software architecture during their and Specifications
second year, for example. LOG430 Software Architecture
LOG515 Software Project Management
The ÉTS is exclusively an engineering school, and LOG550 Real Time System Design
belongs to the University of Québec network of LOG610 Network and Telecommunications
institutions. The school is relatively young, having been LOG625
established in 1974. As of April 2013, the total number of Introduction to Intelligent Systems
active students was approximately 7,000, of which about LOG660 High Performance Database
1,600 are graduate students. It is the fourth largest LOG792 Capstone Project
engineering school in Canada, in terms of the number of
undergraduate students enrolled. The school offers The SE program has been designed to meet the criteria
undergraduate engineering programs in six disciplines, of the Canadian Engineering Accreditation Board, and
including SE, which is the focus of this paper. All the was accredited for the first time in 2001. Graduates of the
engineering programs at the ÉTS integrate cooperative ÉTS are automatically admitted to Quebec’s professional
(co-op) education, which involves practical work carried body of engineers, the Ordre des ingénieurs du Québec
out in industry. Students must complete three co-op terms, (OIQ) (Professional Association of Engineers of the
which last four months each. The school places 2,400 Province of Québec).
students in 1,100 companies each year for these terms.
The students are paid around $11,000 for their 3. SOFTWARE QUALITY
internships, and this constitutes an important revenue ASSURANCE COURSE
source for them during their studies, as well as
establishing the student in an employer/employee 3.1 Lectures
relationship. Because of the incoming students’ technical
background and the co-op nature of our programs, the The SQA course is composed of thirteen 3-hour
ÉTS is considered a very “hands-on” school, which it is, lectures. Each lecture topic is illustrated with industrial
both by design and by mission. It is also worth noting examples, international or professional standards, and
that, according to 2004 statistics, 96% of companies in process improvement model practices. To ensure that
Quebec are small and medium-sized enterprises (SMEs), students grasp the importance of SQA activities, the
and 83% of those employ four people or fewer [11]. This business model concept and the cost of quality concept
reality is taken into account in the design of our programs are stressed throughout the course. When performing
and courses, especially given the fact that Quebec’s SQA activities as part of their term projects, students must
economy includes 25% of Canada’s IT firms [11]. make tradeoffs between prevention, appraisal, conformity,
and rework costs. They must experience firsthand that an
The SE curriculum is a 10-term program, including the investment in prevention and appraisal will significantly
three mandatory 4-month paid internships. Courses are reduce failure costs in the future (rework effort, for
offered during all three 4-month terms of the academic example).
year. Students may opt to complete their internships
during the fall, winter, or summer terms. Every course Since data on the cost of quality published in papers
includes a weekly 3-hour lecture and a weekly 2- or 3- are often very far from the students’ experience, and to
hour laboratory session, where students must complete ensure that the principles associated with that cost are
practical assignments. Table 1 lists the core SE courses in
CEEA13; Paper 109
Montreal, QC; June 17-20, 2013 – 2 of 6 –
Proc. 2013 Canadian Engineering Education Association (CEEA13) Conf.
well understood, students are required to continuously Selection of a review type
measure the cost of rework in their term projects. They 6 Software Audit
are also required to analyze their data and draw Audit and problem resolution according to
conclusions on the benefits of SQA activities. Students ISO/IEC/IEEE 12207
are often amazed that their own project data may reveal a Audit according to the CMMI-Dev model
cost of failure of 50%, and sometimes 70%, of the total Audit according to IEEE-1028
project effort. Corrective actions
The audit and the software quality
The lectures, described in Table 2, follow the assurance plan
sequence set out in the French language textbooks A case study
7 Verification and validation (V&V)
published by the co-authors in 2009 [1,5]. An English Costs and benefits of V&V
version of the textbooks will be published by John Wiley Standards (IEEE-1012) and models which
in late 2013 or early 2014 [6]. require or define V&V
Independent V&V
Table 2: List of SQA course topics Traceability
The validation of software
Lecture Course title Testing
1 Introduction Checklists
How is software quality defined? V&V techniques
Business models and the selection of The V&V plan
software engineering practices 8 Configuration Management
Software errors, mistakes, and failures The usefulness of software configuration
The quality of software management (SCM)
Software quality assurance SCM activities
2 Quality culture IEEE-828 Configuration management
The cost of software quality standard
What is a quality culture? SCM library and branches
The five dimensions of a software project Configuration control
The IEEE Code of Ethics for software Configuration status accounting
engineer Configuration audit
3 Quality requirements The implementation of SCM in a small
Models of software quality (ISO/IEC organization
25000) SCM policy
The definition of software quality 9 Policies, processes, and procedures
requirements Policies
The traceability of requirements in the Processes
software life cycle Procedures
4 Standards and models Organizational standards
Standards, the cost of quality, and Documentation standard (ISO/IEC/IEEE
business models 15289)
An overview of standards and best Case study
practice models The personal software process
The main quality management standards 10 Measurement
ISO/IEC/IEEE 12207 The importance of measurement
IEEE-730 The measurement process of
Other models, standards, guidelines, and ISO/IEC/IEEE 12207
quality procedures The Practical Software and Systems
ISO/IEC 29110 for very small entities Measurement Method
Specific standards for an application ISO/IEC/IEEE 15939 Measurement
domain (e.g. DO-178) standard
ISO/IEC/IEEE 15289 standard for the Measurement in the CMMI-Dev model
description of software products The survey as a measurement tool
5 Software Reviews The implementation of a measurement
Personal review, desk check-type review program
Reviews described in IEEE-1028 and Practical considerations
CMMI for development model The human side of measurement
The Walkthrough 11 Risk management
The Inspection Risk management according to standards
The Project launch review ISO 12207, ISO 9001, and ISO 16326 and
Agile meetings the CMMI-Dev model
Measures ISO/IEC/IEEE 16085 Risk Management
standard
CEEA13; Paper 109
Montreal, QC; June 17-20, 2013 – 3 of 6 –
Proc. 2013 Canadian Engineering Education Association (CEEA13) Conf.
Practical considerations used in a development project and why. They also use the
Human factors in risk management guide as a framework for their team project.
12 Management of suppliers and contracts
Supplier agreement management 3.3 Laboratory Sessions
according to the CMMI-Dev model
Management of external participants The laboratory sessions have been designed in such a
Life cycle of software acquisition way that teams of students can apply the SQA practices
Types of software contracts
Contract reviews presented in the lectures to their SQA term projects.
13 Software quality assurance plan
Introduction Students attend twelve 2-hour laboratory sessions
IEEE-730 and the software quality during a semester. After completing the first two sessions
assurance plan on the code of ethics and the business models, students
embark on a project in teams of four for a period of ten
3.2 Use of Standards weeks where they must apply the SQA practices
presented in the course, using the ISO/IEC 29110
SE students and professors at the ÉTS have access to standard as the framework for the project. The teachers
the full content of the IEEE electronic library, including randomly create the teams, to simulate an industrial
all IEEE standards. These standards are used both in class context where an employee doesn’t usually choose his
and in laboratory sessions. Until recently, we were not teammates.
able to use the ISO standards, as they were too expensive
for students to buy. One of the authors has finalized an At the start of the project, the teams receive a copy of
agreement with the Canadian ISO standards providers: the the Statement of Work (SOW), which they use to develop
Standards Council of Canada (SCC). The agreement the project plan. During the planning phase of the project,
allows all registered SQA students to download standards the four students in a team must share the following roles,
selected by the professor from the SCC website. The as defined in ISO/IEC 29110: analyst, designer,
following standards are discussed in the SQA course: programmer, technical lead, and project manager.
- ISO/IEC/IEEE 24765 (Systems and software Monitoring this process is the responsibility of the teacher
engineering - Vocabulary) or the lab supervisor, who plays the role of customer.
- ISO/IEC/IEEE 12207 (Systems and software
engineering - Software life cycle processes) During the first week of the project, students are also
- ISO/IEC 25000 (Software engineering - Software required to select and install the tools they will use during
product Quality Requirements and Evaluation the project. For example, they must choose and install a
(SQuaRE) - Guide to SQuaRE) document repository, a version control tool, and an issue
- ISO/IEC/IEEE 16085 (Systems and software tracking tool, among others.
engineering - Life cycle processes - Risk
management) Then, the four team members must complete and sign
- ISO 9001 (Quality management systems - a contract which specifies the roles of each participant,
Requirements) the team deliverables, the expectations of each
- ISO/IEC 29110 (Software engineering - Life cycle participant, and the operating rules which they agree to
profiles for Very Small Entities (VSEs)) respect. The course website lists the objectives and
- ISO/IEC 90003 (Software engineering - deliverables for each of the ten weeks of the project. The
Guidelines for the application of ISO 9001:2000 to site also contains all the templates required to produce the
computer software) deliverables. The templates list the content of the
- ISO/IEC/IEEE 15939 (Systems and software documents required by ISO/IEC 29110, such as the
engineering - Measurement process) project plan and the specifications of the software. The
- ISO/IEC/IEEE 15289 (Systems and software site also includes descriptions of the various reviews they
engineering - Content of life cycle information have to perform (e.g. desk check, walkthrough) and forms
products (documentation) for registering any anomalies, e.g. defects, they find
during the review process.
During the course, students are introduced to the
recently published ISO software development standard Teams must estimate the effort that will be needed by
ISO/IEC 29110 [13] targeting Very Small Entities each member to carry out the activities and deliverables
(VSEs). Students use the engineering and management required by ISO 29110. These estimates are recorded on a
guide included in ISO 29110 which is freely available in spreadsheet, and every week members of the team must
English or French from the ISO [3,4], as a framework to record the hours they have worked on defined project
help them understand when software quality practices are
CEEA13; Paper 109
Montreal, QC; June 17-20, 2013 – 4 of 6 –
no reviews yet
Please Login to review.
