Evaluating E-commerce Website Content

Management System in Assisting

Usability Issues
Elida Nurfarihin Abdullah, Suzana Ahmad, Marina ismail, Norizan Mat Diah
Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, 40450 Shah Alam,
Selangor, Malaysia
[email protected], [email protected], [email protected],
2021 IEEE Symposium on Industrial Electronics & Applications (ISIEA) | 978-1-6654-1591-0/21/$31.00 ©2021 IEEE | DOI: 10.1109/ISIEA51897.2021.9509991

[email protected]

Abstract—The Content Management System (CMS) and e- website or e-commerce and, due to their simplicity and ease of
commerce platforms are the website or e-commerce software use, are gaining popularity. According to [6] and [7], CMS has
platforms that are popular for their flexibility and ease of use. risen because frameworks and models make rapid design
However, the usability of a website is one of the most important possible. Content Management System (CMS) is a software
quality factors that is not easily measured as it depends on several system that stores text on the web page and publishes data in a
other factors that are hard to measure. Therefore, this paper aims database instead of as HTML pages [8]. However, when it
to provide a result on which of the compared CMSs is the best for comes to how these tools could assist in solving usability issues
assisting in usability issues and design concepts for a website and design concepts for an e-commerce website, the features
developer. This paper compares several CMSs for e-commerce
that have been implemented into it were catered towards the
websites and lists some of its’ features that may help assist
usability issues and the design concept of an e-commerce website.
user.
The usability principle used in this research is the E-commerce II. BACKGROUND STUDY
Total Quality Management framework (E-TQMF). For this
purpose, we select the most popular e-commerce CMS; namely, Most services offer online product vendors an e-commerce
Shopify, OpenCart, Magento, and WooCommerce. While a set of program that develops the apps to suit the online store's own
website design elements is used when evaluating the design requirements [9]. In today's situation, the purpose of the
concept. The result obtained from this research is the CMS websites of an e-commerce company is to ensure that
features that can help in assisting usability issues and design understudy and other customers are able to purchase and track
concepts. goods as easily as possible and carry out businesses such as
online payment, the saving of last objects, recommendation of
Keywords—E-commerce, Content Management System (CMS), products, etc. Therefore, it is essential to understand the
usability component interests and behavior of users to adapt e-commerce websites to
I. INTRODUCTION the needs of customers [10]. Improving the accessibility of the
website is critical as it can increase revenue, engagement, and
E-business is now considered as an industry standard. The longevity dramatically. Consumer confidence is now regarded
most seasoned and profitable e-commerce firms are beginning as one of the most crucial factors in the success of e-commerce.
to realize that primary performance or loss determinants are not Therefore, website design for e-commerce must be user-centric.
simply an online presence or low price but focus on a high- If a website is misleading, the user would leave and exit from
quality platform. Due mainly to e-commerce, many people the website. A website's performance may rely on the
worldwide prefer to purchase, sell, transfer, or exchange complexity of the application, readability of code, the versatility
products, resources, and knowledge over the Web or via the of code, portability, website linking, and editability [11].
computer network [1]. According to [2], consumer confidence
is listed as one of the most significant growth drivers in e- A. Usability
commerce. Usability is the main concept in human-computer
interaction (HCI). According to ISO 9241-11 criteria, usability
Usability is one of the important things that companies
is defined "As to how effectively, efficiently and happy a
who're offering software products should remember [3], defined
commodity can be used by specific users in a specific context
usability as “a quality attribute that assesses how easy user
of use". ISO 9241-11:1998 focused on usability evaluation by
interfaces are to use." At the same time, ISO 9241, [4] describes
user-based performance, quality, and efficiency evaluation, as
usability as "the degree to which specified consumers could use
this convinced device developers that usability issues exist. [12]
a product to accomplish defined objectives with performance,
stated that it would be important to reduce errors, increase
effectiveness, and satisfaction in a specific application sense."
accuracy and increase the level of trust and optimism on the
Therefore, to plan a successful website, developers must
intended interface by paying due attention to the usability of e-
consider the need of the site's visitors [5]. Many studies focus
commerce portals. As stated by Nielsen (1994) usability is
on usability from the end-user point of view; however, these
broken down into the following goals; effectiveness, efficiency,
studies do not attempt to consider the website's developer
safety, utility, learnability and memorability. The usability
perspectives as the basis of a website quality metric. Therefore,
principles assess the acceptability of interfaces used during
usability is the most important feature so that designers can
heuristic evaluation [13].
define consumers in the early stages and create usability rules
[1]. Lack of usability has critical implications, such as users’
frustration [14]. Usability also refers to the characteristics that
The Content Management System (CMS) mainly facilitates
make it possible, not only during production but also during its
website content management, editing, and improvement. These
operation and maintenance. The implementation of the usability
are electronic applications that enable the introduction of a
guides will minimize the required time and energy in

978-1-6654-1591-0/21/$31.00 ©2021 IEEE

Authorized licensed use limited to: University of Warwick. Downloaded on January 15,2022 at 23:21:17 UTC from IEEE Xplore. Restrictions apply.
undertaking the iterative development steps to more goods, retailers will build an e-commerce store and
"implement/evaluate/improve." A great deal of work has been expand their business. In addition, the built-in SEO feature of
achieved on usability by presenting an overview of various OpenCart helps merchants understand page rankings and
usability testing approaches for web applications. Usability is a keywords to improve their web pages and increase web traffic.
crucial feature of all apps and a significant component of online
shopping portals. [15] defines web usability formally as C. Magento
allowing users to manipulate the functionality of the site to The software platform for Magento eCommerce provides
accomplish a specific goal. online store solutions for large and growing businesses. The
software can be used to manage both front-end and back-end
B. E-Commerce Website Design Concept
operations, creating customer and user experience that is
A good design of an e-commerce website is how friendly branded and tailored. The software connects with many other
the customer perceives when shopping in an online store. programs and offers a variety of enhancements as well as
Usability, reliability, convenience of access, and ease of use maintenance, consultation, and training to help the user develop
made up website design [16]. E-commerce websites are an online store successfully.
distinctive in that designers need to focus on product
presentations within the layout. User interfaces are less D. WooCommerce
complicated but require much other metadata, including prices, WooCommerce is designed for online vendors of small to
quantities, and item types. They are often very complex. large sizes. WooCommerce is an eCommerce plug-in open
Generally, the web design has seven elements which are 1) source for WordPress. The WooCommerce platform enables
context; 2) content; 3) community; 4) customization; 5) any WordPress user to set up a completely free basic online
communication; 6) commerce; and 7) connection [16], [17]. store (additional plug-ins and modules for customization are
C. Content Management System (CMS) charged). Users can create their own plug-ins to add new
features to their online store or purchase from the
Web technology has established the mechanism for WooCommerce shop a pre-made plug-in.
resource sharing through computer networking. HTML, CSS,
XML, JavaScript, asp. NET, and HTTP are among the web E. CMS Features
technologies. Business applications, web servers, databases, Features that should exist in an e-commerce content
and programming languages are essential components of web management system are:
technologies. These web technologies can be used to create
websites. A website is an online toolset such as web pages and • API - A communication mechanism that allows
multimedia content. various applications to exchange data quickly and
Many forms of material can be structured and released smoothly
with the use of CMS, and the style, design, and functionality of • Activity Dashboard - Highlights the information
the Web can be easily and quickly modified as it is built on needed to maximize information by making decisions
prototypes. A CMS is now a preferred choice for those who for the management.
want simple content improvement, streamlined control over a • Automatic Backup - Make a copy of all the
large quantity of content, a wide range of plug-ins, and, above information on the website
all, a way of doing much more, much better, and much more • Campaign Management - A set of activities related
quickly than any other program [8]. It can be updated instantly to an action plan that all lead to a broader specified
and goes live directly if you want to alter the output. Plug-ins corporate objective.
can be used as and when you need to incorporate functionality • Content Filter
and apps. Based on their language or location, specific material • Content Library
can be offered to various users. Compared to static or HTML • Content Management
pages, user engagement can be improved. • Customer Accounts
III. CONTENT MANAGEMENT SYSTEM UNDER STUDY • Customer Segmentation - The process of grouping
customers into groups
This section provides details of the tools that have been • Customizable Branding - Customization of Call
selected for comparisons. They are Magento, WooCommerce,
Bridge and Web Bridge
OpenCart, and Shopify.
• Discount Management
A. Shopify • Drip Marketing - A collection of emails to be sent on
Shopify is a business platform that makes a website for retail a plan to inform and continuously move the user
for everyone to sell. Shopify provides a qualified online down to the recommended intervals.
storefront, credit card acceptance payment service, and retail • Electronic Payments
sales driven by the Shopify POS program. Users get a single • Email Templates
platform to run their company with ease with Shopify's e- • Gift Card System
commerce app. Users can completely configure their online • Multi-Channels Marketing - Function to leverage
store, open up new distribution platforms in seconds, handle
through channels, particularly channels used most by
goods and inventories without restrictions, complete orders in a
the clients.
single phase, monitor distribution and growth trends, and more.
• Multi-Currency
B. OpenCart • Multi-Languages
OpenCart is an e-commerce website open-source that • Multi-Location
retailers use to market and sell online goods. By introducing • Order Management

Authorized licensed use limited to: University of Warwick. Downloaded on January 15,2022 at 23:21:17 UTC from IEEE Xplore. Restrictions apply.
 • Order Processing essential quality metrics, quality attributes, and sub-
• PCI DSS Compliance - A collection of security attributes from both perspectives are measured by
standards to ensure a secure outlook for all businesses using a hierarchical relationship [23]
receiving, processing, storing, or sending credit card 4. WQD (Website Quality Dimensions on Customer
information. satisfaction in the B2C E-commerce Context) is a re-
• Price Management assessed model of ESCM, which leads to customer
• Product Catalogue satisfaction variables 24]
• Product Data Management - Provide a centralized 5. FUX (Facets of User Experience) is conceived to
location for product data that can be obtained from any describe the aspect of UX beyond usability [25]
other departments and channels. 6. ISO (International Organization for Standardization)
• Ratings and Review - To ensure transparency, online is underlying essential software qualities by
and offline drive traffic and sales. addressing several well-known human tendencies that
can affect the delivery and perception of a software
• SEO Management - The role of creating more organic
development project (ISO/IEC, 2015) [26]
traffic from sites such as Google, Bing, and Yahoo.
• Sales Analytics - Analyse the purchasing operation on
“ECommerce Total Quality Management Framework (E-
the platform or app.
TQMF)” has been used to measure quality. One of the reasons
• Sales Order - Specifies the details and costs, quantity,
is that based on the research carried out by [23], the E-TQMF
and terms and conditions of the goods and/or services
considers the Quality side from the consumer's point of view
requested by a particular customer.
and the Customer's Quality Expert. Therefore, a hierarchical
• Sales Reporting - Overview of the status of a business' arrangement is used to calculate the primary metrics, quality
selling activities. features, and characteristics from the user's and quality experts'
• Sales Tax Management - Automation of the sales tax perspectives.
process in the shopping cart
• Search Functionality - An intuitive indexing feature
allows users to search by attributes like publication
dates, keywords, and author, indexing all data to
access them through search functions easily.
• Shipping Management
• Shopping Cart
• Social Media Integration
• Third-Party Integration - Enables seamless external
data integration.
• Website Management - Manage the sites without
requiring users to write thousands of code lines.
• eCommerce Management - Act as an opportunity or
Fig 1. E-commerce TQMF quality measurement tree (Kotian
liability for maintaining an optimized and targeted
& Meshram,2017).
website for the company.
Based on fig. 1, elements such as overall quality assessment,
These listed features could help in assisting in solving the quality characteristics (external and internal), quality sub-goal
identified usability issues. The usability goals highlighted by (quality metrics) and quality characteristics are included in the
[3] are effectiveness, efficiency, safety, utility, learnability, and proposed frameworks. The full quality appraisal is a percentage
memorability, along with other quality metrics and attributes. value achievement that reflects the extent of accumulated
Thus, to determine software quality, a few measurable compliance with quality within the e-commerce system.
construct sets collected are especially related to e-commerce Quality assessment includes the product measured by
service. The example of these construct sets could be seen in consumers and the product specialist team. The basic part of
research conducted by [19] where he listed some of the e- program consistency is a minimum quality metric. The quality
commerce services construct set that exists. The following are sub-goal (quality metrics) has more than one attribute. The
the measurable construct sets. following Table I shows the quality metrics and attributes of
the E-commerce TQMF quality measurement tree model by
1. e-SQ (E-Service Quality) and E-S-QUAL (E-Core [23] that focuses only on usability and the mapping of the tool's
Service Quality) represent essential online service features to the quality metrics model to identify which of the
quality. E-S-QUAL is the core metric of e-SQ, features could assist on the usability issues.
especially E-S-QUAL includes user satisfaction
elements [20]. TABLE I. Metric quality and model attributes [23] and the Tools Features
Mapping
2. ESM (E-commerce Success Metrics) originated from
Information System Success [21]. It is more focused Quality Qualities
Characteristics Features
on commerce traits of a system [22] Metrics Attributes
Usability Ease of use Easy to use and • API
3. E-TQMF (E-commerce Total Quality Management (It describes operate the site • Third-Party
Framework) considers quality aspects both from the the Integration
customer's and quality expert team's side. The effectivenes

Authorized licensed use limited to: University of Warwick. Downloaded on January 15,2022 at 23:21:17 UTC from IEEE Xplore. Restrictions apply.
 Quality Qualities
Characteristics Features A set of questionnaires was completed by 43 respondents
Metrics Attributes
s, User- Designing an easy to • Email Templates
(web developer) with experience in using e-commerce CMS
satisfaction, friendly use and understand • Product tools to build an e-commerce website based on Table III.
and website interface Catalogue
TABLE III. Table Respondents Qualifications
efficiency Simplicity Give simple structure • Order
with which and functionality Management
users • Order
achieve Processing
desired • eCommerce
objectives.) Management
Navigation Supporting web • Search
consumer activity Functionality
Data collected were statistically analyzed then visualized by
User control To permit the user to • Website using Excel and SPSS. E-TQMF framework usability was
always leave the Management adapted.
Web.
Clarity To let the user, know • Activity Magento OpenCart
about its functions Dashboard
Help To provide assistance • Ratings and
function and documentation Review
Understanda To make content • Content Library
bility comprehensible • Content Filter
• Multi-
Languages
Accessibilit To make the Web • Website
y usable to all users Management
Speed To load things • API
quickly
Match real To follow norms of • Multi-Currency
world the real world • Multi-
Languages Shopify WordPress
• Multi-Location
• Ratings and
Review
Learnability To make the program • Product Data
simpler for users (e.g. Management
power, input, output)
Consistency To retain the same • Content
interface features Management
Recognition Simple to recall • Sales Analytics
rather than details
Recall
Aesthetic To create attractive • Content
design and esthetic websites Management
Personalizati Let website be • Website
Figure 3: System Usability Scale for each CMS tools
on customizable Management
Figure 3 shows the results for System Usability Scale for all four
Then, the following table II will show whether the tool's CMS tools selected. Interpretation is referred to as System
features exist suitable to assist on the design concept of an e- Usability Scale Interpretation Score below.
commerce website. Finally, the feature will be compared
System Usability Scale interpretation score
against the seven elements of website design that have been
stated on.
TABLE II. Website design element [23] and the Tools Features Mapping

No. Elements Features

1. • Website Management
Context
• Content Management
2. • Website Management
Content
• Content Management
Mapping between the CMS features and usability quality
3. Community • Ratings and Review attribute along with website design element is also analyzed.
4. Customization
• Content Filter Table IV shows how many of the CMS features exist for each
• Search Functionality of the attributes and elements. Based on the mapping, some of
5. • Order Management
Communication
• Order Processing
the CMS features would apply to several different usability
• Electronic Payment attributes. The number of features for each of the attributes was
• Order Processing as the following Table IV. Each of the usability attributes also
6. Commerce • Price Management has a feature that complies with their characteristics. The
• Shipping Management
• eCommerce Management
mapping between the features of the content management
7. • API system and quality attributes of usability quality metrics may
Connection
• Multi-Channels Marketing conclude that the features of CMS do exist to assist on the
usability issues of an e-commerce website. The CMS for
IV RESULT AND DISCUSSION Magento, Shopify, and OpenCart has a high number of features
that are applied to the usability attribute, so it has a much higher

Authorized licensed use limited to: University of Warwick. Downloaded on January 15,2022 at 23:21:17 UTC from IEEE Xplore. Restrictions apply.
chance to assist on the usability issues of an e-commerce each usability quality metric and elements of a design concept,
website except for Shopify. followed by Shopify, OpenCart, and lastly, WooCommerce.
TABLE IV. Table Number of feature mapped for each attribute
Based on that, Magento as the highest feature quantity could be
said as the best CMSs tool. Also, it can be concluded that the
Number of developer can use the usability quality metrics as a guide for e-
Quality Metrics Qualities Attributes
Features
Usability Ease of use 2
commerce web content management systems design and
User-friendly 2
development.
Simplicity 3 Acknowledgment
Navigation 1
The authors would like to thank the Faculty of Computer
User control 1
and Mathematical Sciences for sponsoring this paper.
Clarity 1
Help function 1
Understandability 3
References
Accessibility 1
Speed 1 [1] R. Yazdanifard, A. Zargar, A. "Today's Need of E-Commerce
Managementto E-Skill Trainings". International Journal of e-Education,
Match real world 4 e-Business, e-Management and e-Learning, vol. 2 no. 1, pp. 52-58, 2012
Learnability 1 [2] M. Maadi and M. Javidnia, "Identification of factors influencing
Consistency 1 building initial trust in e-commerce." Iranian Journal of Management
Recognition rather than Recall 1 Studies, vol .9 no. 3, pp .483-503, 2016
Aesthetic design 1 [3] J. Nielsen, “Usability inspection methods. Conference on Human Factors
in Computing Systems.” CHI '94: Conference Companion on Human
Personalization 1
Factors in Computing Systems, pp.413–414, 1994
The following Figure 3 shows the number of features for [4] Ergonomic Requirements for Office Work with Visual Display
each quality attribute on the chosen tools. Terminals, ISO 9241-11, ISO, Geneva, 1998
[5] T. Singh, S. Malik and D. Sarkar, ”E-commerce website quality
assessment based on usability”. IEEE International Conference on
Computing, Communication and Automation (ICCCA), pp. 101-105,
2016.
[6] S. Wan, D. Li and J. Gao, “Exploring the advantages of content
management systems for managing engineering knowledge in product-
service systems”. Procedia CIRP, 56, pp. 446-450, 2016
[7] F. Bianco and F. Michelino, “The role of content management systems
in publishing firms”. International Journal of Information Management,
vol. 30 no 2, pp. 117-124, 2010
[8] S. K. Patel, V. R. Rathod and J. B. Prajapati, Performance analysis of
content management systems-joomla, drupal and wordpress.
International Journal of Computer Applications, vol. 21 no. 4, pp. 39-43,
2011
[9] W. Ariyani, D. Hanantjo and B. E. Purnama, “E-Commerce Web
Development in Wiga Art”. Int. J. Sci. Res., vol. 4 no. 5, pp. 379-384,
2015
[10] S. Hernández, P. Álvarez, J. Fabra, and J. Ezpeleta, “Analysis of Users’
Behavior in Structured E-Commerce Websites,” IEEE Access, vol 5, pp.
11941-11958, 2017.
Figure 3: Usability Quality Attributes to CMS feature existence.
[11] C. Darie and E. Balanescu, Search engine optimization. Beginning PHP
and MySQL E-Commerce: From Novice to Professional, pp. 189-220,
Figure 3 shows that most of these four CMS analyzed do not 2008.
fully meet the Usability Quality Attributes discussed. For [12] F. Yulin, Q, Israr , S. Heshan, P. Mccole, E. Ramsey and L. Kai, “Trust,
example, the only quality attribute achieved by all four CMS is Satisfaction, and Online Repurchase Intention: The Moderating Role of
'Match Real World', while the rest of the attributes are covers Perceived Effectiveness of E-Commerce Institutional Mechanisms”.
by only one or two of the selected CMS. MIS Quarterly, vol. 38, pp. 407-427, 2014.
[13] E. S. A Majid, N. Kamaruddin and Z. Mansor, ”Adaptation of usability
V CONCLUSION principles in responsive web design technique for e-commerce
development”, 5th Int. Conf. Electr. Eng. Informatics Bridg. Knowl.
The usability of a website is one of the most important between Acad. Ind. Community, ICEEI 2015, pp. 726–729, 2015
quality factors, including information presented in clear and [14] S. J. Hong and K. Y. Tam, "Understanding the adoption of multipurpose
precise form, a lack of ambiguity, and the placement of relevant information appliances: The case of mobile data services”. Information
items in appropriate areas. However, the usability of a website systems research, vol. 17 no. 2, pp. 162-179, 2006
is not easily measured because it is difficult to measure some of [15] T. Powell, Web Design: The Complete Reference, Osborne McGraw-
the factors. This paper highlighted e-commerce CMS features Hill, Berkeley, Calif., 2000.
with the mapping of the features to a usability quality attributed [16] S. Tangchaiburana and K. W. Techametheekul, “Development model of
from the Total Quality Management framework and 7 elements web design element for clothing e-commerce based on the concept of
mass customization,” Kasetsart J. Soc. Sci., vol. 8 no. 3, pp. 242–250,
of website design. The used framework has 16 quality attributes 2017
for e-commerce usability quality metrics. The 36 features listed [17] K. Phakdiwatthanakunand T. Kanjanasuwan, E-commerce. Bangkok,
were collected from a software comparison engine and then Thailand: KTP Comp & Consult. (2004)
mapped to the attributes. Based on the mapping and comparison [18] Compare Magento Commerce vs WooCommerce vs OpenCart vs
of the number of features that exist for each CMSs, it could be Shopify. (n.d.). Retrieved July 12, 2020, from
concluded that Magento has the highest number of features for

Authorized licensed use limited to: University of Warwick. Downloaded on January 15,2022 at 23:21:17 UTC from IEEE Xplore. Restrictions apply.
 https://www.getapp.com/website-ecommerce- [23] H. Kotian and B. B. Meshram, "A framework for quality management of
software/a/magento/compare/woocommerce-vs-opencart-vs-shopify/ e-commerce websites," 2017 International Conference on Nascent
[19] E. Y. Han, (2019). Develop an inclusive UX evaluation framework. B2C Technologies in Engineering (ICNTE), pp. 1-6, 2017
E-commerce heuristic metrics (EHM). Retrieved July 20, 2020 [24] H. Lin, “The Impact of Website Quality Dimensions on Customer
[20] V. A. Zeithaml, A. Parasuraman and A. Malhotra, “Service Quality Satisfaction in the B2C Ecommerce Context”. Total Quality
Delivery through Web Sites: A Critical Review of Extant Knowledge.” Management & Business Excellence, vol. 18 no. 4, pp. 363-378, 2007.
Journal of the Academy of Marketing Science, vol. 30 no. 4, pp. 362-375, [25] P. Morville (2019, October11). User Experience Design. Retrieved from
2002 http://semanticstudios.com/ user_experience_design/ ISO/IEC, 2015).
[21] W. H. DeLone and E. R. McLean, “Information systems success: The [26] ISO/IEC 9126. (2020, January 11). Retrieved from
quest for the dependent variable”. Information Systems Research, 3, pp. https://en.wikipedia.org/wiki/ISO/IEC_9126
60–95, 1992. .
[22] W. H. DeLone and E. R. McLean, “The DeLone and McLean model of
nformation systems success: A ten year update.” Journal of Management .
Information Systems, vol. 9 no. 4, pp. 60–95, 2003.

Authorized licensed use limited to: University of Warwick. Downloaded on January 15,2022 at 23:21:17 UTC from IEEE Xplore. Restrictions apply.