Philippine Journal of Engineering Education, 4(1), Oct 2016, ISSN 0115-3447, pp.

14-19

Implementing an Outcomes-Based Quality Assurance Process for

Program Level Assessment at DLSU Gokongwei College of Engineering

A. Y. Chua1, A. W. C. Oreta2, R. R. Seva3

1
Mechanical Engineering Department, De La Salle University, Manila, Philippines
2
Civil Engineering Department, De La Salle University, Manila, Philippines
3
Industrial Engineering Department, De La Salle University, Manila, Philippines
([email protected], [email protected], [email protected])

Schools, Colleges and Universities Accrediting Agency,

Abstract - This paper describes the process that the Inc. (ACSCU-AAI) (4) Accrediting Agency of Chartered
DLSU-GCOE undertook to institute a Program Level Colleges and Universities of the Philippines (AACCUP),
Outcomes Based Quality Assurance (OBQA) Process. The and (5) Association of Local Colleges and Universities
first stage involved the organization of the GCOE Quality Commission on Accreditation (ALCU-COA) (Ruiz &
Assurance Office (QA)where a Director was assigned to plan
Junio-Sabio, 2012). The accreditation levels provided by
and lead in the process. Each department also identified a
quality coordinator that will manage the programs QA and these organizations are used by CHED in determining
accreditation matters. The second stage was the Centers of Excellence and Centers of Development in
familiarization of the OBE concept and process of all faculty different programs.
members. Seminars and workshops at the college and
departmental level were conducted. The third stage involved In the field of engineering, the Accreditation Board of
the joint effort of all faculty members in the different Engineering and Technology (ABET) accreditation
programs to identify key elements in the OBE framework. provides assurance that the program meets the quality
Finally, the last stage included the gathering of evidences and standards established by the profession. This non-profit
evaluation for further improvement. Some lessons learned
organization is based in the US and is run by professional
were enumerated in this paper after the initial
implementation of the Outcomes Based process. organizations. The focus of ABET is not on compliance but
on continuous improvement.
Keywords Engineering Education, Outcomes-Based
Education, Quality Assurance, Continuous Quality In the Philippines, there are two universities that have
Improvement, Outcomes-Based Assessment been granted ABET accreditation. In 2013, DLSU decided
to pursue a plan for ABET accreditation in order to make
its engineering programs internationally recognized for
I. INTRODUCTION quality. The Gokongwei College of Engineering (GCOE),
on the other hand, believes that the process of continuous
The internationalization of higher education indirectly improvement (CQI) promoted by the ABET framework is
compelled higher education institutions (HEIs) to engage just as important as the actual accreditation. The
in quality assurance. UNESCO defined quality assurance preparation allows the faculty members to assume a critical
as an ongoing, continuous process of evaluating stance on their current teaching practices and student
(assessing, monitoring, guaranteeing, maintaining and learning strategies especially in the method of assessment.
improving) the quality of higher education systems,
institutions or programmes. Recognition refers to the This paper summarizes the program level Outcomes-
acceptance of a foreign certificate, diploma or degree of Based Quality Assurance (OBQA) process [3] developed
higher education as a valid credential by the competent by the Gokongwei College of Engineering with the aim of
authorities and the granting to its holder the same rights improving the performance of the students based on
enjoyed by persons who possess a national qualification for identified Student Outcomes. The paper discusses the
which the foreign one is assessed as comparable [1]. organization of the GCOE Office and the trainings
conducted to further understand the OBE Framework to be
In the Philippines, the Commission on Higher adopted by the college. The identified Program
Education (CHED) is the government body that is Educational Objectives (PEOs) and Student Outcomes
mandated to promote relevant and quality education [2]. (SOs) adopted by the different programs are also explained
There are five accrediting bodies in the Philippines, together with the forms developed to facilitate the
namely; (1) Philippine Accrediting Association of Schools, assessment and evaluation process. Finally, challenges and
Colleges and Universities (PAASCU) (2) Philippine lessons learned during the process are discussed.
Association of Colleges and Universities Commission on
Accreditation (PACU-COA) (3) Association of Christian
 II. PROGRAM LEVEL OBQA PROCESS Outcomes-Based Assessment (OBA) and (iii) Continuous
DEVELOPMENT Quality Improvement (CQI). These three concepts will be
discussed in the succeeding subsections.
This section discusses the process that the GCOE-
DLSU had undergone to institutionalize the OBQA process
in the college. It involved the organization of a GCOE QA
office that championed the OBQA process. It is then
followed by the development of the OBQA framework
together with the assessment and improvement plan for
each program of the college.

A. Organization and Training

There is a need for an office in the college that will

manage all the documents and implementation of the
OBQA process. The establishment of a GCOE QA office
is essential in this case with a GCOE QA Director as the
head. The office was placed in-charge of all accreditation
efforts that will be identified by the different engineering
programs. The structure of the GCOE QA office is shown
in Fig. 1. The GCOE QA office is directly under the Dean
of the college, although it also coordinates with the
university level QA office for synergy of activities and Fig. 2. GCOE OBQA Framework
budgeting. Each engineering program had a Quality
Coordinator that will facilitate the process in the program Outcomes-Based Education (OBE)
level. The Quality coordinators receive honorarium for Outcomes-Based Education is an educational model
their special tasks as QA committee members. in which the curriculum and pedagogy and assessment are
all focused on student learning outcomes [4]. Outcomes-
Based Education is now accepted as a framework in the
accreditation of Engineering Programs. The principles of
Outcomes-Based Education (OBE) can be summarized in
Fig. 3. In OBE, the outcomes are first defined and then the
planning and delivery of all courses in the curriculum
follows. Course design includes syllabus writing and using
a variety of teaching and learning activities (TLAs) and
Fig. 1. GCOE QA Office Structure assessment tasks (ATs) to achieve the outcomes. In
defining the outcomes, a hierarchy must be followed, from
After the organization of the GCOE QA Office, the the university vision-mission at the top, followed by the
GCOE QA Director scheduled and conducted training expected graduate attributes. PEOs, SOs and LOs.
workshops for the Quality Coordinators, Chairs and Vice
Chairs of the different departments. This ensured that each
department has at least three QA champions. After the
training, the quality coordinators were tasked to echo the
training/workshop to their respective departments with the
Chairs and Vice Chairs helping out in the process. With the
training given by people in the same engineering program,
the faculty members were able to relate better to the OBQA
concepts. This also created a sense of ownership among
faculty members in the OBQA process especially when
crafting the specifics of the program level assessment. At
the same time, specific specialization and uniqueness of the
programs can be incorporated in the program level
assessment.
Fig. 3. OBE Framework [5]
B. Framework Development
The expected graduate attributes (EGAs) must be
The Outcomes-Based Quality Assurance Process of consistent with Universitys vision and mission. The EGAs
DLSU GCOE is hinged on three concepts as shown in are characteristics or qualities of students of a university
Fig. 2: (i) Outcomes-Based Education (OBE), (ii) upon graduation. De La Salle University refers to these
attributes as Expected Lasallian Graduate Attributes student
(ELGA). ELGA emphasizes on four qualities Lasallians
should embody upon graduating from the University: (a)
Creative and Critical Thinker, (b) Effective
Communicator, (c) Reflective Life-long Learner, (d)
Service-Driven Citizen.

To achieve the ELGAs, each program in the University

must define a set of Program Educational Objectives
(PEOs). PEOs are broad statements that describe what
graduates are expected to attain within a few years (usually
five years) after graduation. For the GCOE, each
engineering program defined three PEOs which have
common objectives: (a) Leadership in the Profession, (b)
Life-Long Learning, and (c) Social Responsibility. These
PEOs were formulated in consultation with stakeholders
(students, parents, alumni and industry advisers). For
example, the PEOs of the Electronics and Communications
Engineering Program and their alignment to the ELGAs are
Fig. 4. Common Student Outcomes in Engineering Programs
shown in Table 1.
outcome can be classified into three levels: (a) Introductory
TABLE I. PEOs for BSECE Program
or Level 1, (b) Formative, Reinforced, Enabling or Level
PEOs for the Electronics & Communications ELGA 2, and Summative, Emphasized, Demonstrative or Level 3.
Engineering Program A sample curriculum mapping for some courses in the
1 2 3 4
BSCE program is shown in Table II.
1. (Leadership in the Profession) Graduates are
Electronics Engineers occupying positions
requiring highly technical expertise in TABLE II. Curriculum Mapping
X X X
specialized fields of electronics engineering
imbibed with responsible leadership and
ethical values.
2. (Life-long Learning) Graduates demonstrate
life-long learning and continuous professional
X X X
and career development while practicing
their profession.
3. (Social Responsibility) Graduates will
demonstrate service for the community and
society that helps uplift the lives of the X X
people and will manifest concern for the
environment.

To achieve the PEOs, each program must define a set

of Program or Student Outcomes (SOs). The SOs can be
defined by each university or may follow standards
required by national institutions like the Commission on
Higher Education (CHED) or accrediting organizations To be able assess the achievement of SOs,
like the Philippine Technological Council (PTC) in the performance indicators (PI) must be formulated for each
Philippines or ABET. Student outcomes are narrow SO. Performance indicators specify measurable statements
statements that describe outcomes (knowledge, skills, articulating the key characteristics of a student outcome.
abilities, values) of what students are expected to know and These are confirmable though evidence.
be able to do by the time of graduation. For GCOE, the
CHED student outcomes which are similar to the ABET The identified course in the curriculum mapping will
and PTC student outcomes (a to k) were adopted. Fig. 4 be used in the assessment of achievement of the student
shows the general student outcomes which are common to outcomes. Hence, the course syllabi must also follow an
the GCOE engineering programs OBE format. An example of the implementation of an OBE
framework in the course level can be found in related
The next step in OBE is the curriculum mapping of papers of Oreta & Roxas [6]. The key to OBE is alignment
courses with respect to the student outcomes. Each course of the course learning outcomes (LO) and specific
in the curriculum must address at least one student performance indicators of student outcomes. Moreover, the
outcome. The degree in which a course addresses a specific course syllabus must show that the course content, teaching
and learning activities (TLAs), and assessment tasks (ATs) Outcomes-based assessment requires the use of
address the intended course learning outcomes (LO). specific assessment methods that can address the
Felder and Brent [7] presented instructional techniques that performance indicators of student outcomes. Grades are
can be applied in the classroom to effectively achieve the not directly used in assessing the achievement of learning
ABET student outcomes of an engineering program. After outcomes in the course and program level. Because many
course delivery, outcomes-based assessment follows to factors contribute to an assigned grade, it is almost
determine if the course learning outcomes and student impossible to make inferences about what a student knows
outcomes are achieved. or can do by only looking at the grades for a course [8].
There are two ways of assessing the attainment of
Outcomes-Based Assessment (OBA) outcomes: direct and indirect methods. Direct assessment
Outcomes-Based Assessment in the program level is a is based on an analysis of student behaviours or products in
process that involves various steps to verify the attainment which they demonstrate how well they have mastered
of the Student Outcomes. The steps in OBA can be learning outcomes. Direct assessment methods use
summarized as follows: quizzes, exams, and reports to measure students
i. Identify the performance indicators (PI) for each SO performance. Indirect assessment, on the other hand, is
ii. Using the curriculum map, select specific courses per based on an analysis of reported perceptions about student
level that will be used to address the PI mastery of learning outcomes. Examples of indirect
iii. For the courses selected, identify the assessment assessment methods are surveys, interviews, evaluations,
method that will be used for assessing the questionnaire, and focus group. Examples of outcomes-
achievement of the PI based assessment in the course level can be found in the
iv. Prepare the Assessment Plan for each term to guide the literature [9], [10].
faculty
v. Monitor the implementation of the assessment plan To effectively monitor the OBQA process, forms were
designed by the QA director for the use of the QA
Performance Indicators (PI) for each Student Outcome coordinators. The OBQA portfolio management is shown
are specific measureable statements articulating the key in Fig. 5. The OBQA Monitoring Form is regularly
characteristics of a student outcome, confirmable through updated by QA Coordinators and thus informing the QA
evidence. For example, Student Outcome A may have the Director on the status of the implementation of the
following performance indicators: (a) Chooses a assessment plan of each department. The OBQA
mathematical model of a system or process appropriate Monitoring Form is completed at the end of the term when
for required accuracy. (b) Applies mathematical the QA Coordinators submit them after the submission of
principles to achieve analytical or numerical solution to grades. The OBQA Data Gathering Form is updated with a
model equations, and (c) Examines approaches to solving summary prepared by the QA Coordinators. The evidences
an engineering problem in order to choose the more are also submitted to the GCOE QA office for safekeeping
effective approach. and digitizing.

Educational strategies are then identified that will suit

the performance indicators identified and followed by
specific courses used for the assessment considering their
offerings during the academic year. Not all courses will be
used for program level assessment. When all SOs have
their specific PI, specific courses are then selected from the Fig. 5. Portfolio Management
curriculum map that will be used to assess the PI for each
term. This results to the OBQA Assessment Plan. Table III Continuous Quality Improvement (CQI)
is a sample assessment plan for one PI. Each PI of a student To complete the cycle of the OBQA process,
outcome must have a specific assessment plan. Continuous Quality Improvement (CQI) must be practiced
at the course level (Fig. 6) by the faculty and the at the
TABLE III. Assessment Plan program level (Fig. 7) by the department. Every term, a
faculty must assess the achievement of the course learning
outcomes using direct and indirect assessment methods. If
he/she observes an underachievement of the LOs, the
faculty needs to reflect on what possible improvements can
be done with respect to the syllabus, course content, type
of assessment (quiz, exam, project, report), teaching
methods, student-centered learning activities, textbook and
online resources, etc. He/she may have to devise new and
more effective teaching and learning activities such active
learning, cooperative learning, project-based learning,
conduct online tutorials, change the textbook or add more III. CHALLENGES AND LESSONS
problem solving exercises.
To achieve quality education, investment is needed.
The administration must invest by providing funds and
resources to support a CQI program. Incentives must be
given to the faculty especially for those who demonstrate
effective teaching skills and innovative student-centered
learning activities. The faculty must invest time and effort
in learning outcomes-based education principles and must
adopt effective teaching and learning methods. The GCOE
OBQA Program is moving forward because of the
investment provided by both the administration and the
faculty.

Launching a formal method of program assessment

and evaluation was a challenge because faculty members
have preconceived notion about quality assurance such as
being tedious and expensive. There was a need to openly
communicate with the faculty about the issues and tackle
the sources of their negative views. The plan for
Fig. 6 Course Level CQI Cycle
accreditation was presented for comment at the Council of
Chairs. The Dean and the QA Director discussed in detail
Continuous Quality Improvement (CQI) must be the activities to be done by the College, the expected costs
applied every cycle for the program level. One cycle is
and benefits. Differences in opinion about the benefits of a
equivalent to one academic year or three terms. Hence
formal quality assurance framework were settled through
within the Program Level CQI Cycle are Course Level CQI
dialogue between the members of the Council of Chairs
cycles. After one program cycle, the student outcomes and the faculty members of the departments. The
performance data are compiled and compared with the Department Heads assisted the Quality Coordinators in
target standards. In the CQI program level, there should be explaining and implementing the most difficult aspects of
an assessment of all SOs using the different level 3 courses
the program, which is data gathering and analysis.
identified by the program and this is done by the
department or a cluster. If the program assessment shows
Training faculty members on quality management
that the target standards are not achieved, CQI can be empowered them to use the knowledge gained to improve
recommended to the level 3 courses or even to the lower the teaching and assessment process. The success of the
level courses if the department finds some weaknesses in program was hinged on educating the faculty members and
the level 1 and 2 courses. Possible improvements can be
granting them ownership of the process. DLSU trained the
improvement of assessment tasks, revision of the course
faculty to determine what needs to be done and learn the
content, change in prerequisites or even a change of
process on their own with the basic principles in mind.
courses to be assessed. Consultants were not employed in the beginning as they
have a tendency to dictate processes to undertake without
considering the context. Faculty members identified the
processes that will fit their needs so there was less
resistance during the data gathering stage.

Changing a culture is an arduous task for management

because it requires people to veer away from their comfort
zone. The experience in the implementation of OBQA
framework provided the following lessons that other
organizations can benefit from:
i. Culture change can only happen if there is strong
top management support. Faculty members are
very independent-minded and the only way for
them to engage in a difficult task is if there is a
directive from the top. This kind of support
ensures that the activity will have appropriate
investment on the part of management.

ii. Effective and dedicated leaders at the department

Fig. 7 Program Level CQI Cycle level are crucial for successful implementation.
 Faculty members agreed to cooperate but when ACKNOWLEDGMENT
they become busy with examinations and other
teaching-related tasks, there is a tendency to set The authors acknowledge the support and cooperation
aside the required activities for accreditation. The of the GCOE QA Coordinators and the De La Salle
Chair of the Department, in this case, needed to University Administration especially the QA Office in the
constantly remind the faculty of their commitment development of the OBQA program of DLSU-GCOE.
and talk to those who are not performing well.
Incentives for the faculty especially to the QA REFERENCES
coordinators provided the impetus for them to be
dedicated to their assigned tasks. [1] Ruiz, A. J., & Junio-Sabio, C. (2012). Quality Assurance in
Higher Education in the Philippines, Asian Journal of
iii. Faculty members need to be constantly trained on Distance Education, 10(2), 63-70.
different areas especially on effective teaching
[2] Commission on Higher Education. (n.d.). Vision and
and learning methods. Part of the orientation of Mandate. Retrieved June 29, 2015,
new faculty members is the OBQA training so http://www.ched.gov.ph/index.php/home/about-ched/vision-
that they will not be lost in the activities of the mandate/
College. Every trimester, all faculty members of [3] Chua, A. Y., Oreta, A.W.C. and Seva, R.R. An Outcomes-
GCOE had to attend training related to teaching Based Quality Assurance Process for Program Level
and assessment. Assessment at DLSU Gokongwei College of Engineering: A
Guide to QA Coordinators, Unpublished Report, 2015
iv. Conducting regular meetings is a way of showing
commitment to improvement. During these [4] Driscoll, A. & Wood, S. (2007). Outcomes-based assessment
for learner-centered education, Stylus Publishing, Inc.,
meetings, the department heads and Dean had Virginia, USA
been present to listen to reports and inputs of
Quality Coordinators. [5] Oreta, Andres W. C. (2012). Outcomes-Based Education:
As I see it, http://digitalstructures.blogspot.com, January
v. The process is tedious and requires a lot of time 18, 2012
and dedication from faculty members. The leader
must constantly show appreciation openly [6] Oreta, A.W.C. and Roxas, C. (2012). Implementing an
through different channels of communication. Outcomes-Based Framework in the Teaching of Engineering
Mechanics (STATICS)., Proc. International Conference on
CE Education (ICCEE2012), Nov. 9-10, 2012, De La Salle
IV. CONCLUSIONS University, Manila ISSN 2244-3738

This paper presented the general process that GCOE- [7] Felder, Richard and Brent, Rebecca, Designing and
DLSU undertook to create a Program Level Outcomes Teaching Courses to Satisfy the ABET Engineering
Based Quality Assurance (OBQA) Process. The creation Criteria, Journal of Engineering Education, 92 (1), 7-25
of the GCOE Quality Assurance Office together with the (2003)
assignments of Quality Coordinator for each program was
vital in implementation of the process. The faculty was [8] Rogers, G. (2003). Do grades make the grade for program
made familiar to the OBE through the trainings and assessment? Retrieved from www.abet.org
workshops both the college and departmental level. The [9] Gurocak, H. (2009). Direct measures for course outcomes
faculty was involved in the identification of key elements assessment for ABET Accreditation. Paper presented at the
in the OBE framework (e.g. Program Educational 2008 American Society for Engineering Education (ASEE)
Objectives, Performance Indicators, Curriculum Map, and conference. Washington State University-Vancouver.
Assessment Plans) which created a sense of ownership of
the faculty with the OBQA program. A system of gathering [10] Oreta, A. W. C and Roxas, C. (2014)."Outcomes-Based
of evidences was done to make the process of collection Assessment of a Basic Engineering Course (Statics of Rigid
and evaluation easy for the program. Bodies), Proceedings Scholarship and Innovation on
Teaching and Learning (SILT) Symposium, ISSN 2362-
7670, March 7, 2014, p. 39-46, De La Salle University,
The OBQA program of the GCOE-DLSU is just Manila
starting. It is hoped that through the installation of a
structure for continuous quality improvement and the
constant education of all the stakeholders that a truly
unique CQI system will be developed in the university.