SPECIAL SECTION ON NEW TRENDS IN BRAIN SIGNAL PROCESSING AND ANALYSIS

Received October 16, 2018, accepted November 5, 2018, date of publication November 26, 2018,
date of current version December 19, 2018.
Digital Object Identifier 10.1109/ACCESS.2018.2881384

Extending the Technology Acceptance Model for

Use of e-Learning Systems by Digital Learners
AAMER HANIF 1,2 , FAHEEM QAISAR JAMAL1 , AND MUHAMMAD IMRAN 3, (Member, IEEE)
1 Department of Engineering Management, College of E&ME, NUST, Rawalpindi 46000, Pakistan
2 Department of Computer Science, Air University, Islamabad 44000, Pakistan
3 College of Applied Computer Science, King Saud University, Riyadh 11692, Saudi Arabia

Corresponding author: Aamer Hanif ([email protected])

The work of M. Imran was supported by the Deanship of Scientific Research, King Saud University, through the Research Group under
Grant RG-1435-051.

ABSTRACT Technology-based learning systems enable enhanced student learning in higher-education

institutions. This paper evaluates the factors affecting behavioral intention of students toward using e-
learning systems in universities to augment classroom learning. Based on the technology acceptance model,
this paper proposes six external factors that influence the behavioral intention of students toward use of
e-learning. A quantitative approach involving structural equation modeling is adopted, and research data
collected from 437 undergraduate students enrolled in three academic programs is used for analysis. Results
indicate that subjective norm, perception of external control, system accessibility, enjoyment, and result
demonstrability have a significant positive influence on perceived usefulness and on perceived ease of use
of the e-learning system. This paper also examines the relevance of some previously used external variables,
e.g., self-efficacy, experience, and computer anxiety, for present-world students who have been brought up
as digital learners and have higher levels of computer literacy and experience.

INDEX TERMS e-learning, technology-acceptance model, technology-based learning, digital learners,

behavioral intention.

I. INTRODUCTION Classroom allow the integration of instructional materials,

Owing to the internet and mobile-based multimedia com- assessment modules, collaboration tools, and these systems
munication technologies, students today have a multitude of have become a key constituent of instruction delivery in the
opportunities to learn from the same material usually taught university system [3]–[5]. Advantages gained with e-learning
in classrooms and requiring physical presence. In a develop- systems in education provide quick access to course materials
ing country like Pakistan, novel technologies are providing and information, communication, collaboration, and many
new ways for students to engage in learning activities and different ways to learn based on student needs. As new
enabling them to use these for educational purposes thereby technological innovations keep surfacing, their applications
augmenting student learning in addition to regular classroom in higher education pave the way for additional research.
education. The value of technology in education for prepar- However, the primary objective of the adoption of new tech-
ing engaging lectures, particularly by using Information and nologies and studies on exploring their enabling factors are
Communication Technology (ICT) tools for collaborative always important. Accordingly, the issue of implementing
work and adopting self-regulated strategies has a positive technology-based instruction as a norm or standard practice
impact on course effectiveness for students [1]. in universities becomes a question of technology acceptance
Handheld smart devices are common nowadays. They as to what motivates students to accept and use new tech-
allow students to keep in touch with course materials and nology for learning. However, students of present times have
explore additional resources for learning. Peer interaction grown up with technology around them and are called digital
using Web 2.0 tools enables students to undertake real-time natives because they handle technology in a natural way [6].
discussions, learn from mistakes, synchronize their thoughts, These digital learners have peculiar characteristics such as
and motivates them to use internet resources with the end preference for a technology-infused learning environment
result of an enriched learning experience [2]. The availabil- with flexible schedule, need for collaborative learning with
ity of e-learning systems such as Blackboard and Google immediate feedback, and learning through activity with a
2169-3536
2018 IEEE. Translations and content mining are permitted for academic research only.
VOLUME 6, 2018 Personal use is also permitted, but republication/redistribution requires IEEE permission. 73395
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 A. Hanif et al.: Extending the Technology Acceptance Model for Use of e-Learning Systems by Digital Learners

strong inclination toward the use of mobile devices and social II. RELATED WORK
networks [7]. Therefore, the question of technology accep- Rapid advancements in ICT are resulting in the massive
tance by digital natives or learners requires the incorporation integration of smartphone applications and social networks
of a host of individually related influential factors relevant to into the personal and professional lives of users of these tech-
this generation of technology. nologies. With the development of technology, various mod-
The objectives of this research study are to develop and els for technology-acceptance behaviors started to emerge.
present a model of e-learning system adoption based on Users’ acceptance of these technologies and resulting mod-
the technology-acceptance model (TAM) and subsequent els and theories of technology acceptance have been widely
research experiences gathered from different variants of the explored [10]–[13]. Table 1 provides a summary of various
model. We explore and develop relationships between the models that have been proposed in literature. The external
perceived usefulness (PU) and perceived ease of use (PEOU) factors proposed and considered significant in influencing BI
of the e-learning system in the presence of certain influ- to use technology systems are also provided. Most of these
encing factors, including learners’ self-efficacy, enjoyment, models are targeted toward the use of information systems in
perception of external control, subjective norm, result demon- various contexts and indicate the importance of research on
strability and system accessibility. After a review of the BI to use these systems.
literature relevant to these internal and external variables,
an extension to the TAM model is proposed to study the TABLE 1. Models and theories of technology acceptance.
behavioral intention (BI) of students toward the adoption of
e-learning systems by students. The hypothesized relation-
ships are analyzed and conclusions are drawn based on the
findings.
To the best of our knowledge, research that links tech-
nology acceptance behavior with characteristics of digital
natives or learners is lacking. Although many previous studies
have explored and extended TAM, it has primarily been used
on the basis of frequently occurring external variables in
literature [8]. Hence, some of these studies have reported
insignificant relationships between external variables and the
use of technology considering that the proposed models have
been tested on samples comprising digital natives [9]. These
facts provide the motivation for this research and highlight
a need for re-investigating external factors and relationships
among TAM variables. Our research contribution culminates
in an extended TAM in which external factors relevant to
digital natives are proposed. Frequently studied external fac-
tors in the past such as computer anxiety, experience, and
self-efficacy are questioned for relevance to digital natives,
resulting in our proposed model that has better explanatory
power compared with previous models wherein these vari-
ables were included. Moreover, the external factor of attitude
toward using technology was initially removed from TAM
and its successor models because of its nonsignificant effects
on TAM relationships. However, attitude toward technology
has been reinstated in our proposed model with noteworthy
research findings.
This paper is organized as follows. The next section intro-
duces the work related to various models and theories of The development of new theoretical research frameworks
technology acceptance. An overview of technology accep- have mostly relied on TAM and its variants as it became
tance specifically for learning in higher education institutes the most widely studied model. This model is supported by
is described in Section III. Section IV gives an account of the abundant empirical data collected and analyzed in various
external variables and presents the proposed extended model contexts. The key variables frequently used in the mentioned
for technology acceptance for e-learning systems. Section V models can be identified as PU (performance expectancy),
mentions the research methodology, and Section VI pro- PEOU (effort expectancy), subjective norm (social influ-
vides the results of the measurement and structural models. ence), enjoyment, and attitude toward the use of technology.
Section VII discusses the results, and Section VIII concludes The next section discusses the application of TAM to the
the paper with implications of the research. acceptance of e-learning systems in higher education.

73396 VOLUME 6, 2018

A. Hanif et al.: Extending the Technology Acceptance Model for Use of e-Learning Systems by Digital Learners

III. TECHNOLOGY ACCEPTANCE FOR LEARNING IV. THEORETICAL MODEL CONSTRUCTS

The TAM initially proposed by Davis is a well-known entity This study proposes an extended model based on TAM
for explaining how human attitude and behavior predict the by providing a relationship among three major constructs,
use of technology in the presence of other external vari- namely, the PEOU, PU, and BI of using an e-learning system.
ables [15]. For technology-based learning systems, a number Although variable ATT was initially a part of the TAM, it was
of additional variables require consideration. These variables removed because of the weak role between PU and ATT. The
include the cognitive, social, and personal characteristics of argument was that users without a positive attitude can still
learners, and these characteristics play vital roles in the design use a technology if its perceived benefits were visible [15].
and employment of these systems [23]. Thus, technology The ATT construct was not included in the GETAMEL model
acceptance is influenced by a number of factors requiring for similar reasons [9]. However, we decided to include atti-
exploration. In this regard, TAM provides two key beliefs, tude in our model to test student attitude toward e-learning
namely, PU and PEOU that explain the use of a technology systems and to determine the strength of this construct in our
system after being influenced by user attitude (ATT) and BI. local higher education context. Fig. 1 depicts the proposed
First, PU is the degree to which a learner believes using model that extends TAM by adding six external variables
technology increases his or her learning. Second, PEOU is and attitude construct to the model. The rationale for their
related to the belief that technology based learning is free of inclusion is described below:
intellectual effort [15]. TAM application has been extensively
explored in the higher education context to determine how
students PU and PEOU affect their acceptance of e-learning
initiatives [24]–[27].
TAM was used to study e-learning acceptance in Saudi
Arabia, and PU, PEOU, and subjective norm (SN) are
observed to predict BI [28]. In the same study, PU was
predicted by PEOU, subjective norm, and job relevance,
whereas PEOU was predicted by computer self-efficacy, per-
ceptions of external control, computer anxiety, and perceived
enjoyment. Many external factors affecting BI and the use
of technology have been previously proposed. The most
recent addition to the TAM family is the generalized TAM
(GETAMEL) based on five external factors proposed after
analyzing 107 research studies from the past 10 years explor-
ing different external variables on the subject [8]. GETAMEL
incorporates self-efficacy (SE), subjective norm (SN), experi- FIGURE 1. An extended technology acceptance model for e-learning.
ence (EXP), computer anxiety (CA), and enjoyment (ENJ) as Hypothesized relationships are indicated.

the five most common external factors affecting technology

acceptance for e-learning. GETAMEL has been tested and
validated for e-portfolios used by students, and the external A. RESULT DEMONSTRABILITY (RES)
variables CA and EXP are not found to have a significant A positive attitude about a system’s usefulness is seen when
relationship with PU and PEOU contrary to past findings [9]. the use of that system produces positive results for the
All these findings provide a rationale for the present research, user. This factor is denoted as result demonstrability and
enabling our key motivation for dropping these two constructs is defined as the tangibility of outcome of using the inno-
and exploring additional variables that can explain the BI of vation [30]. This construct has been included in TAM-2,
using e-learning systems by digital natives. and user perceptions of result demonstrability have been
The topic of e-learning system adoption with regard to found significant [19]. Subsequent studies have included
TAM is rarely studied in developing countries such as result demonstrability as an external variable and reported
Pakistan. Only one related study conducted in Pakistan con- its positive influence on PU [28], [31]–[34]. Students tend
siders ‘‘student readiness’’ as the single student-related exter- to accept entities that can produce positive results for them,
nal factor in the proposed model [29]. In spite of being a therefore, we included this variable in the proposed model.
developing country, Pakistan has sufficient internet penetra- The following hypothesis is stated for testing.
tion with over half of its population comprising of mobile [H1] A positive relationship exists between RES and the
phone users. The younger generation and university students PU of an e-learning system.
among the population are extensive users of social networks.
Hence, the present study attempts to establish a model that B. SUBJECTIVE NORM (SN)
has been validated in the context of a developing country and Subjective Norm is about influencing a person’s percep-
is deemed applicable to other countries as well with similar tion or thinking by people close to him regarding the per-
technology related characteristics. formance of a certain behavior [20]. This factor is part of

VOLUME 6, 2018 73397

 A. Hanif et al.: Extending the Technology Acceptance Model for Use of e-Learning Systems by Digital Learners

the theory of planned behavior, TAM-2, and UTAUT models, E. PERCEPTION OF EXTERNAL CONTROL (PEC)
as evident in Table 1. In the case of e-learning, it is about External control is defined as the extent to which a person
how a student’s inclination to use e-learning is influenced trusts that relevant technical resources exist in an organization
by the opinions of friends and faculty members in an edu- to support system utilization for performing tasks [20]. PEC
cational context. Subjective norm can be the extent to which has been alternatively called facilitating conditions because
a student feels the environment and peer pressure to practice complex systems require organizational support for their suc-
e-learning [35]. People who are valued commonly influence a cessful implementation. The same has been considered as
person in real life; likewise, if beliefs can be used as an argu- an important external factor in previous research, leading
ment in favor of e-learning adoption, then a student would to increased levels of user acceptance of new systems [19],
definitely give importance to such beliefs and render them [20], [46]. Moreover, during the development of TAM-3,
useful for adoption. Subjective norm then acts like an intrin- PEC has been identified as a determinant of the PEOU of
sic motivator to influence students to use e-learning [36]. an e-learning system [21]. In a previous research on testing
A student is more likely to develop affirmative beliefs about TAM-3 in Saudi Arabia to ascertain learner intentions of
technology-based learning and its applications in real life if using the e-learning system, facilitating conditions or percep-
he or she would be positively influenced by those close to tions of external control have been determined to have the
him or her. Considering that peer and instructor influences strongest effect on the PEOU [28]. This finding suggests that
exist in local culture, SN is included in the model and the the beliefs of an individual about the presence of necessary
following hypothesis is proposed. resources and technical or organizational support enable the
[H2] A positive relationship exists between SN and the PU use of the system. For these reasons, we decided to include
of an e-learning system. this variable in the proposed model. The related hypothesis is
stated as follows.
C. ENJOYMENT (ENJ) [H5] A positive relationship exists between PEC and the
Enjoyment in the context of technology-based systems is PEOU of an e-learning system.
related to the activity of using these systems and perceiving
them to be agreeable and pleasing on their own [25]. Multiple F. SYSTEM ACCESSIBILITY (SYSACC)
studies involving research on multimedia e-learning sys- System accessibility simply implies that an accessible system
tems, web-based training, and learning systems have shown a can be used more conveniently and frequently than a system
strong positive relationship between the enjoyment and PU of that is inaccessible and provides barriers in its use [24].
these learning systems, thereby increasing students’ intention Problems such as the unavailability of appropriate technical
to use these systems [28], [37]–[40]. Moreover, a significant infrastructure and slow speed internet connections hinder
positive relationship has been found between enjoyment and system accessibility. Students tend not to use online learning
PU in 100% of studies proposing the GETAMEL model [8]. materials when issues of network connection, internet speed,
Thus, this external variable is considered a strong candidate and access reliability exist [47], [48]. Accordingly, these tech-
for inclusion in the hypothesized model of this study, and the nical aspects of accessibility become critical success factors
following hypothesis is proposed. that determine the usability of an online-learning system.
[H3] A positive relationship exists between ENJ and the System accessibility has been considered as a significant
PEOU of an e-learning system. external factor in other studies that had based their research
model on TAM to predict the adoption and use of e-learning
D. SELF-EFFICACY (SE) systems [24], [36], [49], [50]. Given that poorly accessible
Self-efficacy is used as a common external factor of TAM systems may not be usable even in the presence of favorable
in a large number of studies. Computer self-efficacy (CSE) attitude and intention to use, this variable is included in the
is defined as one’s belief about his/her ability to accomplish proposed model along with the following hypothesis to be
a particular task using a computer [41]. Computer literacy tested.
and computer anxiety are related in the sense that they can [H6] A positive relationship exists between SYSACC and
affect users’ self-efficacy. Higher computer anxiety may the PEOU of an e-learning system.
result in poor performance and will negatively influence
the use of computers because these are avoided by peo- G. ADDITIONAL HYPOTHESES
ple who consider them too complex and believe that they Apart from the six external variables discussed earlier, addi-
cannot use them [42]. This finding suggests that students tional paths exist between the key constructs of TAM (Fig. 1),
who have higher e-learning self-efficacy are more likely to which need to be tested for significance after the inclusion of
use e-learning and computer-supported education [43]–[45]. these external variables. Thus, the following five additional
Thus, the following hypothesis is proposed. hypotheses are proposed.
[H4] A positive relationship exists between SE and the [H7] A positive relationship exists between the PEOU and
PEOU of an e-learning system. PU of an e-learning system.

73398 VOLUME 6, 2018

A. Hanif et al.: Extending the Technology Acceptance Model for Use of e-Learning Systems by Digital Learners

[H8] A positive relationship exists between the PEOU of the most common disciplines in undergraduate education
and ATT toward an e-learning system. in Pakistan. Demographic details of participants and their
[H9] A positive relationship exists between the PU of and e-learning experiences were gathered in addition to informa-
ATT toward an e-learning system. tion about constructs of the research model. The approach
[H10] A positive relationship exists between the PU of and used for research constituted two steps. First, the measure-
BI toward an e-learning system. ment model was validated by confirmatory factor analysis
[H11] A positive relationship exists between the ATT and (CFA). Second, a structural model and path analysis were
BI toward an e-learning system. used to explore the relationships among the constructs to
perform hypothesis testing. The software used for statistical
V. RESEARCH METHODOLOGY analysis were IBM SPSS version 24 and AMOS [51], [52]
This study used a quantitative approach with a cross-sectional
design to investigate the relationships among the constructs B. DEMOGRAPHIC ANALYSIS
of the proposed research model. The development of research A total of 479 responses were collected. Among them,
instruments involved 10 constructs adapted from previously 437 were considered usable after thorough data screening,
validated instruments used in similar contexts. Table 2 which included the removal of un-engaged responses and
contains references to previous research from where the multivariate outliers. The sample consisted of 76% males
questionnaire items for each of these constructs were adopted. and 24% females. About 87% of the students had their own
Each construct had multiple items measured using the five- PC/laptop. More than 50% of the students had computer
point Likert scale ranging from (1) strongly disagree to experience of over 7 years, and only about 15% students had
(5) strongly agree. computer experience of less than 3 years. About 50% of the
students stated that they used the internet for over 14 hours
TABLE 2. Convergent validity and reliability of constructs. per week for learning activities. The most common way of
learning outside the classroom was using YouTube videos and
from related lectures of other universities offered at websites
such as Coursera. From the demographic data, we concluded
that these students were thoroughly experienced in using
computers and the internet for learning purposes, and that
computer anxiety was not an issue worth exploring in this
sample.

VI. RESULTS AND ANALYSIS

This section reveals the analysis results. Assumptions of
linearity and normality of constructs were established prior
to the statistical analysis of collected data. Reliability and
validity checks of the scale for dataset size N=437 were con-
ducted. Analysis results of the measurement model, structural
model and path analysis for the hypotheses developed for this
study are presented as follows.

A. MEASUREMENT MODEL
The measurement model was tested using AMOS 24 software
by conducting CFA with maximum likelihood estimates.
Validity and reliability of the constructs were checked first.
Composite reliability (CR) values should be >0.70, and aver-
age variance extracted (AVE) values should be >0.50 for con-
vergent validity [55]. Reliability of the research instrument
was determined by Cronbach’s Alpha coefficient method
and should be >0.70 for acceptability [56]. Table 2 shows
that CR values are all >0.75 and AVE values are all >0.5,
indicating that all constructs have no issues related to con-
A. SAMPLING AND PROCEDURE vergent validity and reliability of scale. Measurement model
Data was collected from undergraduate students of twelve fit indices are provided in Table 3, in which a combination
universities in the Islamabad/Rawalpindi region and divided of absolute and incremental fit indices are shown. These
into two strata (public and private sector institutions). are the most frequently reported indices in literature related
Students studying computer science, management science, to structural equation modeling (SEM) [56]. The model-fit
and engineering disciplines were included as these were indices given in Table 3 are found to be within the required

VOLUME 6, 2018 73399

 A. Hanif et al.: Extending the Technology Acceptance Model for Use of e-Learning Systems by Digital Learners

TABLE 3. Model fit measures. measurement model. Structural model GOF closer to mea-
surement model-fit values are desirable and suggest a better
fit. Table 3 provides model-fit indices for the hypothesized
structural model.
The standardized coefficients are reported in Table 4, along
with the results of hypothesis testing. The structural model
with path coefficients are presented in Fig. 2. The fit indices
for the hypothesized model suggest adequate fit as the GOF
statistics for the model are well within the acceptable limits
of a good model fit.

TABLE 4. Parameter estimates and results of hypotheses.

threshold values [57]. These results indicate that the proposed

theoretical model is a good fit with observed data gathered
through the survey.

B. STRUCTURAL MODEL
The structural model attempted to identify dependence rela-
tionships among the model constructs, as the relationships are
assigned from one construct to another based on the research
model that has been proposed. A two-step approach was used
to assess the structural model as suggested by [58]. First,
goodness-of-fit (GOF) indices for the structural model are
evaluated, after which standardized parameter estimates are
used to justify the causal relationships and test the proposed
hypotheses.
The first step requires testing of the overall model GOF
and assessing it using a similar criterion as done for the

FIGURE 2. Structural model with path estimates.

73400 VOLUME 6, 2018

A. Hanif et al.: Extending the Technology Acceptance Model for Use of e-Learning Systems by Digital Learners

TABLE 5. Comparison of goodness of fit measures with competing model.

C. COMPARISON WITH COMPETING MODEL habits of students to the extensive use of social networks
Based on the results of hypothesis testing, a competing model and multimedia systems for communication. Hence, they also
is considered for comparison purposes. The idea was to enjoy using these systems for learning at their own pace
identify the effect of model relationships with the variable and in their own time because multimedia e-learning systems
SE removed from the model. This model called Model-A provide students with a gratifying learning experience. This
contains all original constructs except for SE given the non- experience increases their motivation to learn even outside the
significant effect of SE on the PEOU. Model-fit measures formal classroom setting.
for our proposed extended TAM and Model-A are provided SE does not have a significant relationship with the PEOU
in Table 5 for comparison. contrary to previous research findings [28], [44].
Table 5 shows that the competing model with SE removed This finding is unexpected as it appears to contradict com-
results in a better model as evident from model-fit measures. mon belief that individuals with higher levels of computer
An interesting observation is that model R2 dropped by 1% SE are bound to be confident about their use of computer
only in the model without SE, indicating negligible reduction systems and can overcome any difficulty related to com-
in model explanatory power. puter use. Some previous studies as highlighted by [8] have
reported similar findings [35], [64], [65]. The most likely
VII. DISCUSSION explanation of this inconsistency is the fact that students in
This study aims to determine the relationship between pro- these studies have higher levels of computer experience and
posed external factors and the BI toward using e-learning efficacy. Results of the competing model without SE also
platforms in education as explained by their PU and PEOU. indicate that this construct does not affect BI toward using
Table 4 shows support for most of the proposed hypotheses. technology. Students who are digital natives and proficient
The results of the proposed model’s GOF measures confirm in using internet resources have a higher SE by default.
that the proposed model can adequately represent the col- Therefore, this construct does not contribute significantly as a
lected data and help understand the BI of digital learners determinant for the PEOU, neither does it significantly affect
toward using e-learning systems. The significance of each of BI when removed from the model. However, these findings
the model constructs as determined by hypothesis testing is may be substantiated by undertaking additional research in
then discussed. diverse contexts and with different student experience levels
RES had significant positive influence on PU as with technology.
hypothesized, which was consistent with previous studies The PEC positively affects PEOU of the e-learning system.
[28], [59], [60]. The tangibility of outcome or results of This construct is important as it relates to the availability of
using e-learning systems seem to appear significant to under- necessary technical infrastructure in the organization to sup-
graduate students who recognize that learning from internet port use of the system. Even if students strongly intend to use
resources can increase their learning and enables them to bet- e-learning systems, their BI is affected by the strong or poor
ter understand the course material. This finding also suggests availability of the necessary technical and support features of
that digital learners tend to go for technological resources for the system. This finding is also consistent with past studies
learning when they can evidently benefit from these resources [21], [28], [60].
by investing their time in learning from their use. SYSACC has a positive effect on the PEOU of an
The role of SN as an extrinsic motivational factor affecting e-learning system. This finding is consistent with past studies
student attitude and BI toward using e-learning has been pre- [66], [67]. With the advent of new technologies and learn-
viously established [9], [24], [28], [61]. Peers and instructors ing opportunities available, system accessibility to learners
are in a position to shape and influence student perceptions also requires the availability of asynchronous communication
to adopt e-learning systems. Moreover, social pressure and between learners and instructors, discussion fora, evaluation
influence of important persons in one’s life are prevalent fac- mechanisms, and system-support mechanisms for students.
tors in the Pakistani society, wherein individuals feel obliged The availability of mobile devices with students and the
to act upon or even change their behavior, opinion, and atti- provision of Wi-Fi services in university campuses is signif-
tude under the influence of those who they feel are important icantly increasing easy and quick access to information.
in their lives. PEOU does not have a significant relationship with
The positive effect of ENJ on PEOU is consistent with PU contrary to previous findings [8], [9]. A recent study
many past reports [9], [37], [62], [63]. The reason for the using GETAMEL to study e-learning adoption in Azerbaijan
significance of this construct is the relationship of the current showed that PEOU had a non-significant effect on the

VOLUME 6, 2018 73401

 A. Hanif et al.: Extending the Technology Acceptance Model for Use of e-Learning Systems by Digital Learners

PU [68]. About 70% of the students in this study had com- computer anxiety and experience, which are tested and found
puter and mobile related experience of over 3 years and were significant in other models such as GETAMEL, previously
proficient in using technological resources for e-learning. seem to be of little relevance with changed characteristics of
In the present research, students are also experienced in the students in the present age. Given that learning technologies
use of computers and the internet. Given that PEOU reflects keep evolving and newer ways to deliver learning content are
how users assess their ease of use of e-learning and deals bound to emerge, continuous research is required to update
with the intrinsic motivational aspect of using information the TAM. The proposed model is a step in that direction and is
technology [69], students with higher experience levels of expected to generate additional research in different contexts.
using computers have other reasons (e.g., enjoyment, and
result demonstrability) to use e-learning systems. For these VIII. CONCLUSIONS AND IMPLICATIONS
students or digital natives, using technology is a current way The deployment and use of e-learning systems in Pakistani
of life. universities is increasing, and students are expected to use
We observed that PEOU had a significant positive effect them in their courses. An extended TAM model for the
on ATT; however, the path from the PU to ATT was found adoption of technology for e-learning is proposed with fac-
to be non-significant. These findings are consistent with tors considered relevant to digital learners in a developing
previous ones from studies involving the attitude construct country. Subjective norm, perception of external control, sys-
[24], [70], [71]. Students finding e-learning systems easy to tem accessibility, enjoyment, and result demonstrability have
use have a favorable attitude toward using the system. a significant positive influence on using e-learning systems.
The PU is also found to have a significant positive affect Learners’ self-efficacy does not seem to influence the use
on BI. This path has been found to be significant in many of these systems by digital learners. Computer experience
previous studies as it is one of the key paths in the original and anxiety are not relevant to digital natives. Students are
TAM and has been a part of many extended TAM-based likely to use the e-learning systems if they have complete
models [8], [68], [70]. This finding shows that without any access, can see tangible results of use, are socially influenced
attitude formation, a system perceived useful can find a strong appropriately, and enjoy using these for a perceived benefit
BI toward using it. Useful implications for educators can be in fulfilling their academic needs. The role of ‘‘attitude’’
providing assistance to learners and emphasizing the effec- construct is re-established, and its importance is thus justi-
tiveness and usefulness of e-learning systems to increase their fied. This rationale explains the selection and testing of the
usage. Moreover, information and training sessions can help external/behavioral variables in the proposed model.
students understand how they could improve their learning in The practical implications of this research are relevant to
academic courses by using online resources. administration and faculty as well in addition to students.
This research has found attitude to be a significant predic- University administration can invest in relevant technical
tor of BI toward using e-learning. Some studies had removed infrastructure to enable successful implementation of
ATT from the model because of its weak role between inter- e-learning systems in academic programs. The use of mobile
construct relationships [40], [72]. However, the present study technologies is rendering e-learning systems convenient to
finds significant roles of the ATT construct, i.e., its effect on use and promotes collaborative learning. Faculty can prepare
BI is greater than that of PU, consistent with previous studies course material and assignments to enable personal mobile
[73], [74]. These findings suggest that a student’s attitude is devices to expand student engagement by exploring blended
a strong predictor of BI toward using the system with the fact learning opportunities. Research on technology acceptance
that students find this system easy to use. However, system for e-learning in the Pakistani higher education context is
usefulness does not affect student attitude toward it, thereby lacking, and this study helps shape similar studies undertaken
highlighting the importance of student attitude in Pakistani in other developing countries.
higher education. BI toward using the e-learning system is In the dynamic technical and academic learning scenarios,
more strongly affected by positive attitude toward the system no model can be the ultimate explanation of ground reality.
than by the usefulness of the system. Thus, students with a Hence, the present study is also limited in the sense that it
positive attitude toward e-learning systems are more likely can generate extensive results based on additional dimensions
to use them. Hence, course leaders and instructors need to being added to explain student BI and participation. Further
shape student attitudes and improve their engagement and research can be undertaken to study other external factors
desirability to use the system apart from merely proving that in different contexts and cultures in developed countries as
the system is useful for their studies and learning. well. The mediating and moderating effects on the BI toward
The key contribution of this research is an adaptation of technology adoption also warrants further analysis to study
TAM that explains the BI toward using e-learning systems user acceptance of e-learning.
by adding attitude and eliminating experience and computer
anxiety constructs, which do not apply to digital natives. The REFERENCES
model explains 64% of the variation (R2 ) in the dependent [1] V. Venkatesh, A.-M. Croteau, and J. Rabah, ‘‘Perceptions of effectiveness
of instructional uses of technology in higher education in an era of Web
variable (BI), which is greater than the 58% variation 2.0,’’ in Proc. IEEE 47th Hawaii Int. Conf. Syst. Sci. (HICSS), Jan. 2014,
explained by GETAMEL [9]. Constructs like self-efficacy, pp. 110–119.

73402 VOLUME 6, 2018

A. Hanif et al.: Extending the Technology Acceptance Model for Use of e-Learning Systems by Digital Learners

[2] F.-T. Leow and M. Neo, ‘‘Redesigning for collaborative learning [26] A. M. Elkaseh, K. W. Wong, and C. C. Fung, ‘‘Perceived ease of use
environment: Study on students’ perception and interaction in and perceived usefulness of social media for e-learning in Libyan higher
Web 2.0 tools,’’ Procedia-Social Behav. Sci., vol. 176, pp. 186–193, education: A structural equation modeling analysis,’’ Int. J. Inf. Edu.
Feb. 2015. Technol., vol. 6, no. 3, pp. 192–199, 2016.
[3] Y.-H. Lee, Y.-C. Hsieh, and Y.-H. Chen, ‘‘An investigation of employees’ [27] S. K. Sharma, J. K. Chandel, and S. M. Govindaluri, ‘‘Students’ acceptance
use of e-learning systems: Applying the technology acceptance model,’’ and satisfaction of learning through course websites,’’ Educ., Bus. Soc.,
Behav. Inf. Technol., vol. 32, no. 2, pp. 173–189, 2013. Contemp. Middle Eastern Issues, vol. 7, nos. 2–3, pp. 152–166, 2013.
[4] S. Bhat, R. Raju, A. Bikramjit, and R. D’Souza, ‘‘Leveraging E-learning [28] S. S. Al-Gahtani, ‘‘Empirical investigation of e-learning acceptance and
through Google classroom: A usability study,’’ J. Eng. Educ. Transforma- assimilation: A structural equation model,’’ Appl. Comput. Inform., vol. 12,
tions, vol. 31, no. 3, pp. 129–135, 2018. no. 1, pp. 27–50, 2016.
[5] S.-C. Lin, S. F. Persada, and R. Nadlifatin, ‘‘A study of student behavior [29] S. Iqbal and I. A. Qureshi, ‘‘M-learning adoption: A perspective from a
in accepting the blackboard learning system: A technology acceptance developing country,’’ Int. Rev. Res. Open Distrib. Learn., vol. 13, no. 3,
model (TAM) approach,’’ in Proc. IEEE 18th Int. Conf. Comput. Supported pp. 147–164, 2012.
Cooperat. Work Design (CSCWD), May 2014, pp. 457–462. [30] G. C. Moore and I. Benbasat, ‘‘Development of an instrument to measure
[6] M. Prensky, ‘‘Digital natives, digital immigrants part 1,’’ Horizon, vol. 9, the perceptions of adopting an information technology innovation,’’ Inf.
no. 5, pp. 1–6, 2001. Syst. Res., vol. 2, no. 3, pp. 192–222, 1991.
[7] N. Sarkar, W. Ford, and C. Manzo, ‘‘Engaging digital natives through social [31] P. E. Ramírez-Correa, J. Arenas-Gaitán, and F. J. Rondán-Cataluña, ‘‘Gen-
learning,’’ Systemics, Cybern. Inform., vol. 15, no. 2, pp. 1–4, 2017. der and acceptance of e-learning: A multi-group analysis based on a
[8] F. Abdullah and R. Ward, ‘‘Developing a general extended technology structural equation model among college students in chile and spain,’’ PLoS
acceptance model for E-learning (GETAMEL) by analysing commonly ONE, vol. 10, no. 10, p. e0140460, 2015.
used external factors,’’ Comput. Human Behav., vol. 56, pp. 238–256, [32] V. Lefievre, ‘‘Gender differences in acceptance by students of training
Mar. 2016. software for office tools,’’ presented at the 14th Annu. Int. Conf. Educ.,
[9] F. Abdullah, R. Ward, and E. Ahmed, ‘‘Investigating the influence of the 2012.
most commonly used external variables of TAM on students’ perceived [33] J. Arenas-Gaitán, F. J. Rondán-Cataluña, and P. E. Ramírez-Correa, ‘‘Gen-
ease of use (PEOU) and perceived usefulness (PU) of e-portfolios,’’ Com- der influence in perception and adoption of e-learning platforms,’’ in Proc.
put. Human Behav., vol. 63, pp. 75–90, Oct. 2016. 9th WSEAS Int. Conf. Data Netw., Commun., Comput., 2010, pp. 30–35.
[10] Y. H. Al-Mamary, M. Al-Nashmi, Y. A. G. Hassan, and A. Shamsuddin,
[34] N. Zhang, X. Guo, and G. Chen, ‘‘IDT-TAM integrated model for IT
‘‘A critical review of models and theories in field of individual acceptance
adoption,’’ Tsinghua Sci. Technol., vol. 13, no. 3, pp. 306–311, 2008.
of technology,’’ Int. J. Hybrid Inf. Technol., vol. 9, no. 6, pp. 143–158,
[35] Á. F. Agudo-Peregrina, Á. Hernández-García, and F. J. Pascual-Miguel,
2016.
‘‘Behavioral intention, use behavior and the acceptance of electronic learn-
[11] W. Jen, T. Lu, and P.-T. Liu, ‘‘An integrated analysis of technology
ing systems: Differences between higher education and lifelong learning,’’
acceptance behaviour models: Comparison of three major models,’’ MIS
Comput. Human Behav., vol. 34, pp. 301–314, May 2014.
REVIEW, Int. J., vol. 15, no. 1, pp. 89–121, 2009.
[36] S. Y. Park, M.-W. Nam, and S. B. Cha, ‘‘University students’ behavioral
[12] P. C. Lai, ‘‘The literature review of technology adoption models and
intention to use mobile learning: Evaluating the technology acceptance
theories for the novelty technology,’’ J. Inf. Syst. Technol. Manage., vol. 14,
model,’’ Brit. J. Educ. Technol., vol. 43, no. 4, pp. 592–605, 2012.
no. 1, pp. 21–38, 2017.
[13] M. Marangunić and A. Granić, ‘‘Technology acceptance model: A litera- [37] H. Zare and S. Yazdanparast, ‘‘The causal Model of effective factors
ture review from 1986 to 2013,’’ Universal Access Inf. Soc., vol. 14, no. 1, on intention to use of information technology among payamnoor and
pp. 81–95, 2015. traditional universities students,’’ Life Sci. J., vol. 10, no. 2, pp. 46–50,
2013.
[14] M. Fishbein and I. Ajzen, Belief, Attitude, Intention and Behav-
ior: An Introduction to Theory and Research. Reading, MA, USA: [38] K. Praveena and S. Thomas, ‘‘Continuance intention to use facebook:
Addison-Wesley, 1975. A study of perceived enjoyment and TAM,’’ Bonfring Int. J. Ind. Eng.
[15] F. D. Davis, ‘‘Perceived usefulness, perceived ease of use, and user accep- Manage. Sci., vol. 4, no. 1, pp. 24–29, 2014.
tance of information technology,’’ MIS Quart., vol. 13, no. 3, pp. 319–340, [39] S. H.-P. Shyu and J.-H. Huang, ‘‘Elucidating usage of e-government
1989. learning: A perspective of the extended technology acceptance model,’’
[16] I. Ajzen, ‘‘The theory of planned behavior,’’ Org. Behav. Human Decision Government Inf. Quart., vol. 28, no. 4, pp. 491–502, 2011.
Process., vol. 50, no. 2, pp. 179–211, 1991. [40] Y.-C. Chen, Y.-C. Lin, R. C. Yeh, and S.-J. Lou, ‘‘Examining factors
[17] R. L. Thompson, C. A. Higgins, and J. M. Howell, ‘‘Personal computing: affecting college students’ intention to use Web-based instruction systems:
Toward a conceptual model of utilization,’’ MIS Quart., vol. 15, no. 1, Towards an integrated model,’’ Turkish Online J. Educ. Technol., vol. 12,
pp. 125–143, 1991. no. 2, pp. 111–121, 2013.
[18] E. M. Rogers, Diffusion of Innovations, 4th ed. New York, NY, USA: [41] H. V. Kher, J. P. Downey, and E. Monk, ‘‘A longitudinal examination
The Free Press, 1995. of computer self-efficacy change trajectories during training,’’ Comput.
[19] V. Venkatesh and F. D. Davis, ‘‘A theoretical extension of the technology Human Behav., vol. 29, no. 4, pp. 1816–1824, 2013.
acceptance model: Four longitudinal field studies,’’ Manage. Sci., vol. 46, [42] C.-L. Lee and M.-K. Huang, ‘‘The influence of computer literacy and com-
no. 2, pp. 186–204, 2000. puter anxiety on computer self-efficacy: The moderating effect of gender,’’
[20] V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis, ‘‘User accep- Cyberpsychology, Behav., Social Netw., vol. 17, no. 3, pp. 172–180, 2014.
tance of information technology: Toward a unified view,’’ MIS Quart., [43] V. Celik and E. Yesilyurt, ‘‘Attitudes to technology, perceived computer
vol. 27, no. 3, pp. 425–478, 2003. self-efficacy and computer anxiety as predictors of computer supported
[21] V. Venkatesh and H. Bala, ‘‘Technology acceptance model 3 and education,’’ Comput. Educ., vol. 60, no. 1, pp. 148–158, 2013.
a research agenda on interventions,’’ Decision Sci., vol. 39, no. 2, [44] C.-S. Chang, E. Z.-F. Liu, H.-Y. Sung, C.-H. Lin, N.-S. Chen, and
pp. 273–315, 2008. S.-S. Cheng, ‘‘Effects of online college student’s Internet self-efficacy on
[22] V. Venkatesh, J. Y. L. Thong, and X. Xu, ‘‘Consumer acceptance and use learning motivation and performance,’’ Innov. Educ. Teach. Int., vol. 51,
of information technology: Extending the unified theory of acceptance and no. 4, pp. 366–377, 2014.
use of technology,’’ MIS Quart., vol. 36, no. 1, pp. 157–178, 2012. [45] N. Pellas, ‘‘The influence of computer self-efficacy, metacognitive self-
[23] M. Siadaty and F. Taghiyareh, ‘‘E-learning: From a pedagogical perspec- regulation and self-esteem on student engagement in online learning pro-
tive,’’ Int. J. Inf. Sci. Manage., vol. 6, no. 2, pp. 99–117, 2012. grams: Evidence from the virtual world of second life,’’ Comput. Human
[24] S. Y. Park, ‘‘An analysis of the technology acceptance model in understand- Behav., vol. 35, pp. 157–170, Jun. 2014.
ing university students’ behavioral intention to use e-learning,’’ J. Educ. [46] S. Taylor and P. A. Todd, ‘‘Understanding information technology usage:
Technol. Soc., vol. 12, no. 3, pp. 150–162, 2009. A test of competing models,’’ Inf. Syst. Res., vol. 6, no. 2, pp. 144–176,
[25] Y. Park, H. Son, and C. Kim, ‘‘Investigating the determinants of con- 1995.
struction professionals’ acceptance of Web-based training: An extension [47] M. A. Musa and M. S. Othman, ‘‘Critical success factor in e-learning: An
of the technology acceptance model,’’ Automat. Construction, vol. 22, examination of technology and student factors,’’ Int. J. Adv. Eng. Technol.,
pp. 377–386, Mar. 2012. vol. 3, no. 2, pp. 140–148, 2012.

VOLUME 6, 2018 73403

 A. Hanif et al.: Extending the Technology Acceptance Model for Use of e-Learning Systems by Digital Learners

[48] W.-C. Poon, K. L.-T. Low, and D. G.-F. Yong, ‘‘A study of Web-based [70] R. Boateng, A. S. Mbrokoh, L. Boateng, P. K. Senyo, and E. Ansong,
learning (WBL) environment in Malaysia,’’ Int. J. Educ. Manage., vol. 18, ‘‘Determinants of e-learning adoption among students of developing coun-
no. 6, pp. 374–385, 2004. tries,’’ Int. J. Inf. Learn. Technol., vol. 33, no. 4, pp. 248–262, 2016.
[49] Y.-H. Lee, C. Hsiao, and S. H. Purnomo, ‘‘An empirical examination of [71] A. Meerza and G. Beauchamp, ‘‘Factors influencing undergraduates atti-
individual and system characteristics on enhancing e-learning acceptance,’’ tudes towards ICT: An empirical study in Kheis,’’ Turkish Online J. Educ.
Australas. J. Educ. Technol., vol. 30, no. 5, pp. 562–579, 2014. Technol., vol. 16, no. 2, pp. 35–42, 2017.
[50] B. Wu and C. Zhang, ‘‘Empirical study on continuance intentions [72] M. A. Al-Hawari and S. Mouakket, ‘‘The influence of technology accep-
towards E-learning 2.0 systems,’’ Behav. Inf. Technol., vol. 33, no. 10, tance model (TAM) factors on students’ e-satisfaction and e-retention
pp. 1027–1038, 2014. within the context of UAE e-learning,’’ Educ., Bus. Soc., Contemp. Middle
[51] J. Gaskin and J. Lim, Master Validity Tool, AMOS Plugin, Eastern Issues, vol. 3, no. 4, pp. 299–314, 2010.
2nd ed. Gaskination’s Statwiki, 2016. [Online]. Available: [73] Y. J. Kim, J. U. Chun, and J. Song, ‘‘Investigating the role of attitude in
http://statwiki.kolobkreations.com/ technology acceptance from an attitude strength perspective,’’ Int. J. Inf.
Manage., vol. 29, no. 1, pp. 67–77, 2009.
[52] J. Gaskin and J. Lim, Model fit Measures, AMOS Plugin,
[74] I. Arpaci, ‘‘Understanding and predicting students’ intention to use mobile
2nd ed. Gaskination’s Statwiki, 2017. [Online]. Available:
cloud storage services,’’ Comput. Human Behav., vol. 58, pp. 150–157,
http://statwiki.kolobkreations.com/
May 2016.
[53] V. Venkatesh, ‘‘Determinants of perceived ease of use: Integrating control,
intrinsic motivation, and emotion into the technology acceptance model,’’
Inf. Syst. Res., vol. 11, no. 4, pp. 342–365, 2000.
[54] R. Cheung and D. Vogel, ‘‘Predicting user acceptance of collaborative
technologies: An extension of the technology acceptance model for e- AAMER HANIF received the B.E. degree in
learning,’’ Comput. Educ., vol. 63, pp. 160–175, Apr. 2013.
aerospace engineering from NED University,
[55] L. T. Hu and P. M. Bentler, ‘‘Cutoff criteria for fit indexes in covariance Karachi, in 1989, the M.S. degree in computer
structure analysis: Conventional criteria versus new alternatives,’’ Struc-
science from FAST-NU, Islamabad, in 2006, and
tural Equation Model., Multidisciplinary J., vol. 6, no. 1, pp. 1–55, 1999.
the M.S. degree in engineering management from
[56] J. F. Hair, Jr., W. C. Black, B. J. Babin, R. E. Anderson, and R. L. Tatham,
CASE-UET, Taxila, in 2010. He is currently pursu-
Multivariate Data Analysis, vol. 6. Upper Saddle River, NJ, USA: Prentice-
ing the Ph.D. degree with the Department of Engi-
Hall, 2006.
neering Management, College of E&ME, NUST,
[57] J. B. Schreiber, A. Nora, F. K. Stage, E. A. Barlow, and J. King, ‘‘Reporting
structural equation modeling and confirmatory factor analysis results:
Pakistan. He is also working as a Faculty Member
A review,’’ J. Educ. Res., vol. 99, no. 6, pp. 323–338, 2006. with the Department of Computer Science, Air
[58] E. W. L. Cheng, ‘‘SEM being more effective than multiple regression University, Islamabad, Pakistan. His research interests include data min-
in parsimonious model testing for management development research,’’ ing and analytics, project management, and technology enhanced learning.
J. Manage. Develop., vol. 20, no. 7, pp. 650–667, 2001. He has published international conference papers in these areas.
[59] N. P. Wingo, N. V. Ivankova, and J. A. Moss, ‘‘Faculty perceptions
about teaching online: Exploring the literature using the technology accep-
tance model as an organizing framework,’’ Online Learn., vol. 21, no. 1,
pp. 15–35, 2017.
[60] M. Y. Yi, J. D. Jackson, J. S. Park, and J. C. Probst, ‘‘Understanding FAHEEM QAISAR JAMAL received the master’s
information technology acceptance by individual professionals: Toward an and Ph.D. degrees in engineering management
integrative view,’’ Inf. Manage., vol. 43, no. 3, pp. 350–363, 2006. from the University of Engineering and Technol-
[61] T.-H. Chu and Y.-Y. Chen, ‘‘With good we become good: Understanding ogy, Taxila, Pakistan, in 2007 and 2014, respec-
e-learning adoption by theory of planned behavior and group influences,’’ tively. He has been an Associate Professor with the
Comput. Educ., vols. 92–93, pp. 37–52, Jan./Feb. 2016. Department of Engineering Management, CEME,
[62] Y. S. Poong, S. Yamaguchi, and J.-I. Takada, ‘‘Investigating the drivers of NUST, Islamabad, Pakistan, since 2015. His
mobile learning acceptance among young adults in the world heritage town research interests include organizational behavior,
of Luang Prabang, Laos,’’ Inf. Develop., vol. 33, no. 1, pp. 57–71, 2017. organizational change, organizational psychology,
[63] A. S. Al-Adwan, A. Al-Madadha, and Z. Zvirzdinaite, ‘‘Modeling stu- human resource management, supply chain man-
dents’ readiness to adopt mobile learning in higher education: An empirical agement, and project management.
study,’’ Int. Rev. Res. Open Distrib. Learn., vol. 19, no. 1, pp. 1–21, 2018.
[64] S. H. Purnomo and Y.-H. Lee, ‘‘E-learning adoption in the banking work-
place in Indonesia: An empirical study,’’ Inf. Develop., vol. 29, no. 2,
pp. 138–153, 2013.
[65] M. Rezaei, H. M. Mohammadi, A. Asadi, and K. Kalantary, ‘‘Predicting
MUHAMMAD IMRAN has been an Assistant
e-learning application in agricultural higher education using technology
Professor with the College of Applied Computer
acceptance model,’’ Turkish Online J. Distance Edu., vol. 98, no. 1,
pp. 85–95, 2008. Science, King Saud University (KSU), since 2011.
[66] R. Thomson, C. S. Fichten, A. Havel, J. Budd, and J. Asuncion, ‘‘Blend-
He has published a number of research papers
ing universal design, E-learning, and information and communication in top international conferences and journals. His
technologies,’’ Universal Design in Higher Education: From Principles research interest includes mobile and wireless
to Practice. Cambridge, MA, USA: Harvard Education Press, 2015, networks, Internet of Things, software defined
pp. 275–284. networking, cloud and edge computing, and infor-
[67] W. S. Shin and M. Kang, ‘‘The use of a mobile learning management mation security. He has been involved in more than
system at an online university and its effect on learning satisfaction 50 conferences and workshops in various capaci-
and achievement,’’ Int. Rev. Res. Open Distrib. Learn., vol. 16, no. 3, ties such as a chair, co-chair, and technical program committee member.
pp. 110–130, 2015. He has received number of awards such as the Asia Pacific Advanced
[68] C.-T. Chang, J. Hajiyev, and C.-R. Su, ‘‘Examining the students’ behavioral Network Fellowship. He also serves as an Associate Editor for the IEEE
intention to use e-learning in Azerbaijan? The general extended technol- Access, the IEEE Communications Magazine, Future Generation Computer
ogy acceptance model for E-learning approach,’’ Comput. Edu., vol. 111, Systems, Wireless Communication and Mobile Computing Journal (SCIE,
pp. 128–143, Aug. 2017. Wiley), Ad Hoc & Sensor Wireless Networks Journal(SCIE), IET Wireless
[69] D. Gefen and D. W. Straub, ‘‘The relative importance of perceived ease of Sensor Systems, and the International Journal of Information Technology
use in is adoption: A study of E-commerce adoption,’’ J. Assoc. Inf. Syst., and Electrical Engineering.
vol. 1, no. 1, pp. 1–28, 2000.

73404 VOLUME 6, 2018