Science Arts & Métiers (SAM)

is an open access repository that collects the work of Arts et Métiers Institute of
Technology researchers and makes it freely available over the web where possible.

This is an author-deposited version published in: https://sam.ensam.eu

Handle ID: .http://hdl.handle.net/10985/23181

To cite this version :

Marc SAWIROS, Ruding LOU, Maged RAWASH - NEXT-GEN E-COMMERCE in the METAVERS
- In: 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-
Adjunct), Singapour, 2022-10-17 - 2022 IEEE International Symposium on Mixed and Augmented
Reality Adjunct (ISMAR-Adjunct) - 2022

Any correspondence concerning this service should be sent to the repository

Administrator : [email protected]
 NEXT-GEN E-COMMERCE in the METAVERS
Marc Sawiros¹ Ruding Lou² Maged Rawash³
Arts et Metiers Institute of Technology, Arts et Metiers Institute of Technology, Megalodon360
LISPEN, HESAM Université, UBFC LISPEN, HESAM Université, UBFC

. Two months after Facebook announced it, the term "Metaverse"

ABSTRACT began to be used. A search of the Factiva database turned up
The Metaverse's goal is to create a virtual world that more than 12,000 news articles in English in the past two months,
incorporates augmented reality, virtual reality, 3D holographic fewer than 4,000 in the first nine months of 2021 and fewer than
avatars, and other forms of communication. In this paper, a new e- 400 in previous years (surprisingly, Facebook was the most
commerce experience was developed to contain the new mentioned company in the article, nearly 10 times more than
technology in user friendly applications that can be used in the Microsoft, the next company mentioned.) Meanwhile, Google
Metaverse as a facility to combine online trading and the new Trends shows that the word has a large number of searches
technology of the virtual reality. concerning the “Metaverse”, About 20 times higher from mid-
October. One of the biggest sectors that was affected by this new
Keywords: E-commerce, 3D-scanner, mesh cloud, point Clouds, trending technology was the commerce sector, and as a continue
segmentation, classification, photogrammetry, application to the idea of E-commerce which first start as “Boston Computer
developing. Exchange” which launched all the way back in 1982. It was
primarily an online market that served people who wanted to sell
Index Terms: K.6.3 [Management of Computing and Information their used computers. Passing through all the online shopping
Systems]: Software Management—Software development; K.7.0 websites (applications such as Amazon, Alibaba, etc...) which
[The Computing Profession]: General. were aiming to affect the market growth of all the commerce rate
around the world and also facilitate the process of buying and
1 INTRODUCTION selling. Every shop now days have an online site which aims to
simply the process of buying or even watching the products from
VR has definitely caught people's attention in recent years.
your 2D screen. With the new era of the “Metaverse” the
Being a new user interface paradigm that offers many advantages
technology of the virtual reality is taking a few steps toward the
in many application areas, also providing easy, powerful and
traditional 2D platform experience and many companies started to
intuitive way of human-computer interaction. User can view and
get involved in the world of virtual shopping experiences, as
manipulate a simulated environment without any need to learn
mentioned before, the “Metaverse” was improving a lot of sectors,
how the complicated user interface works. Hence many
and one of these sector is the E-commerce sector in which the
applications such as aircraft Simulators, architectural exercises or
“Metaverse” will take a huge part in constructing the future of
data visualization systems are relatively well developed. More
virtual sopping due to the abilities of visualizing spaces,
recently, VR has been used as a remote collaboration environment
interacting with virtual models and connecting with the world
in the recreation, educational and social area. Also a big sector of
throw your headset.
industrial work is now facilitated due to the new technology and
drifting a lot of huge worldwide companies to enter this virtual
The application aims to improve the normal virtual tours and
world due to the amount of advantages that they can find in this
turn it into a more integrated and interactive application for a
technology. better user experience. The main idea is to replace
modeling/drawing/designing an artificial shop, the reverse
As written in the article “A short history of the Metaverse” engineering and 3D scan is used for digitalizing a real shop. A
Bernard Marr mentioned that the term “Metaverse” was first used mesh cloud that contains millions of points captured by a 3D
in Neil Stevenson's 1982 novel, Snow Crash. Stevenson's scanner can be used as an environment for the user‟s avatar.
“Metaverse” was a virtual place where characters could go to Using this environment to integrate the user in a realistic virtual
escape a dreary totalitarian reality. In the early 1990s, Sega world that can be accessible throw any VR headset. User can walk
introduced VR arcade machines like the SEGA VR-1 motion throw different spaces that are presented as a realistic captured
simulator, which users enjoyed in many arcades. On October 28, mesh cloud points and interact with the products to create a fully
Mark Zuckerberg announced that Facebook would change its immersive shopping experience. Using the VR headset controllers
name to “Meta” to reflect a new focus on creating the to move, hover, rotating and selecting.
“Metaverse”, a vast, immersive online empire that encompasses
the entire digital society and economy. Facebook (aka. “Meta”) With this application, user can visit a whole shopping mall using
sees the Oculus virtual reality headset as an industrial portal. Just a VR headset and also explore items and even complete all
like today's smartphones are connected to the mobile internet. purchasing steps starting from exploring and specifying
specifications of the product to trying the production ending with
the payment and shipping details and checking out.

(1) e-mail: [email protected]

(2) e-mail: [email protected]
(3) e-mail: [email protected]
 which affected the customer service support and also affected the
feelings of users inside the virtual world.

Figure 1: Realized application screenshot.

In the Metaverse, digital experiences that are very realistic will

be created, unlike the 2D interface of traditional e-commerce Figure 2: Alibaba VR application.
experiences. The Metaverse has the potential to bridge the gap
between e-commerce and the physical shopping experience. A 3 PROPOSED APPROCH
recent study from “Shopkick” found that 70% of consumers Taking advantage of the technology of 3D scanners to collect
believe they can try, touch and see physical products are the most accurate data of the surrounding space and export it in the form of
important aspects of shopping. Creating an online platform that is point cloud to integrate the user as a character in the virtual space
flexible and modifiable to allow the application to improve and using a VR headset and allowing the user to move around virtual
learn more about the users and also be able to expand and add spaces and interacting with different products and be able to
more spaces to collect the biggest amount of data to help users watch, spectate and buy products throughout the VR headset. This
access and connect with more virtual places that exist in real life could be the E-commerce experience in the Metaverse that could
(as shown in figure 1) without the need to 3D model and also build the future of online trading for the next generations.
giving the user the complete six degree of freedom that allows the
user to move in the 3 axis of movement and rotate in the 3 axis of 3.1 3D scanning VS 3D modeling
rotation in any scanned space. Using artificial intelligence can 3D modeling is the process of developing a mathematical
help in segmentation of the mesh cloud captured by the 3D representation of a surface of an object in three dimensions using
scanner to help developer select products in the environment software that manipulates edges, vertices, and polygons in a
without having to manually cut each product from the simulated 3D space. Three-dimensional models represent a
environment to add the required scripts and animation to improve physical body by using a collection of points in 3D space,
the interface and the application sequence and simplify the
connected by various geometric entities. 3D modeling consumes
animated steps for all type of users making the application
lots of time and limit the quality of the output because of the lack
considered as user friendly for all ages above 12 years old.
of image reality and accuracy of the rendered output. 3D scan
2 STATE OF THE ART gives more accurate textures and level of details making the
output closer to reality than the 3D rendered model, it also much
The Metaverse is at this point not hypothetical; it's inside the easier to use 3D scanners than using 3D modeling software. Time
strategy for realizing itself and a result of reforming the E- consumption is also a huge aspect that should be considered as a
commerce scene with the shift from "add to cart" to "add to barrier in 3D modeling. On the other hand an individual 3D scan
room". As a matter of fact, E-Commerce inside the Metaverse
usually takes between 1.5 to 7 minutes.
offers the final word joining between conventional physical stores
and internet shopping. It will be intriguing to perceive how more
In this research, 3D scanning was the optimum solution to
stores embrace Metaverse innovation and furthermore the manner
in which purchasers adjusts to it. The long run of E-commerce collect the maximum amount of spaces in the form off cloud point
inside the Metaverse offers vast potential outcomes, benefits, and and moving to the experience developing phase.
detriments. By and by, the present moment and long haul impacts
the Metaverse will wear society at large still ought to be 3D Laser scanners rely on trigonometric triangulation to
completely investigated by architects and organizations. accurately capture a 3D shape in the form of millions of points.
More precisely, they work by projecting a point or a laser line
Alibaba is one of the foremost effective companies in China onto an object (explained in firgure3) and then recording its
when it comes to VR. In 2016, it set up its possess VR Lab called reflection with sensors. Since the sensors are at a known distance
“GnomeMagic” Lab and accounted its VR technique within the from the laser source, accurate point measurements can be made
future. In Alibabab‟s VR shop Purchase+ , clients can choose in by calculating the angle of reflection of the laser light. Knowing
case they need to go shopping within the US or in Australia (as the distance from the scanner to the object, the scanning hardware
shown in figure2). Another highlight from Alibaba is VR can map the surface of the object and then acquire a 3D scan. This
installment. In VR shops, clients can pay the charge essentially by method is called triangulation because the laser dot (or line),
“head nodding”. The negative part about this application was sensor and laser emitter form a triangle as shown in the image
mainly about the isolation of the virtual world and not interacting below. There are many types of laser scanners on the market, you
with any human being or even interacting with a virtual character can choose from portable, desktop or professional/industrial
devices. The key is that they work in a short amount of time.
 3.3.1 Application interface
Developing an easy interface with big and clear buttons that has
fixed locations allowing the user to easily pick their choice
between different choices. Simplifying the controllers of the VR
headset by adding ray cast that guide the user and demonstrate
their hand motion. After choosing the avatar, an animated human
model will appear the present the user in the Metaverse and will
act as the user‟s body during the E-commerce experience making
the user feel more immersed in the scanned virtual space.

Figure 3: Laser 3D scanner.

Structured light 3D scanners technology also uses trigonometric

triangulation, but it works by projecting a light pattern onto the
object to be scanned and not through a laser line (or point).
The pattern is projected onto the object using an LCD projector or
other stable light source (as seen in figure4). One or more sensors
(or cameras), slightly offset from the projector, observe the shape
of the light beam and calculate the distance from each point of the Figure 5: Avatar choosing.
field of view. The textured light used in the scanning process can
be white or blue, and the light pattern is usually a series of stripes, 3.3.2 6DoF (six degree of freedom)
but it can also be a dot matrix or other shape. This type of device, Alludes to the particular number of axes that an unbending
like the laser scanner, works at close range and can be handheld or body is able to unreservedly move in three-dimensional space. It
mounted on a tripod. characterizes the number of free parameters that characterize the
setup of a mechanical framework. Particularly, the body can move
in three measurements, on the X, Y and Z axes, as well as alter
orientation between those axes in spite of the fact that revolution
as a rule called pitch, yaw and roll (see figure6).

Figure 4: Structured light 3D scanners.

3.2 Point cloud segmentation

Segmentation is the process of grouping point clouds in
multiple homogenous regions with similar properties, while
classification is the next step. Edge-based segmentation
algorithms have two main stages: edge detection to outlines the
borders of various regions and grouping of points inside the
Figure 6: Six degree of freedom diagram.
boundaries to deliver the ultimate segments. Edges during a given
depth map are defined by the points where changes within the 4 REALIZED PROTOTYPE
local surface properties exceed a given threshold. The mostly used
local surface properties are normal, gradients, principal curvatures A test application was developed as a prototype to be the first
or higher order derivatives. application based on 3D scanning for an E-commerce mall; this
mall is located in Cairo, Egypt. We will discuss the different
3.3 Integrating avatars stages of developing this application, the final result, the
reflection on the market and the E-commerce trading in the
After collecting the required data for the environment using the
Metaverse world. The application consist of a welcome interface
3D scanner, developing avatars that represent the user in the
created using “Unity” which allows the user to select his character
virtual world in a TPS (third person shooter) game mode, giving
and then can select which store he wants to virtually visit, and by
the user the freedom to choose their gender and their looks to
clicking on the button responsible for the specific store it will
integrate the user in the application and making it more immersive
automatically immerse the user in the 3D scanned mesh cloud that
(as shown in figure5). Developing the movement and the rotation
is ready to act as an environment for the user and will be ready to
of the character in the 6Dof (six degree of freedom).
contain the user‟s avatar and immerse the user in the virtual shop
as if he was physically inside.
 The starting point was “Tradeline” one of the first to introduce 4.2 Exporting point cloud files
Apple Authorized Resellers to the Egyptian market where you can Matterport allows to download MatterPak™ Bundle, which is a
buy all Apple products. The store has dimension of 15m*8m set of exportable and downloadable assets, generated from
which required around 36 scans (marked in figure7 & 8) to cover Matterport 3D models, that designers, architects, engineers, and
the first floor of the store. construction professionals can use to jumpstart design projects or
a can be imported into third-party programs (such as 3Ds max,
ReCap, Revit, or AutoCAD). This method did not bring the best
quality of the 3D scan but it ws more practical to test the
application using this pack due to the small size of OBJ file which
was 625MB. Using “MeshLab” to filter the cloud point (see
results in figure10) and trim the unnecessary meshes.

Figure 7: Store scanned elevation.

Figure 10: Point cloud with connected meshes.

4.2.1 Segmentation
Using “CloundCompare” automatic segmentation tool, many
trial were made focusing on the segmentation of products from
their stands and their surroundings to give the mesh cloud a
unique separated meshes that are more practical in the use of the
application for the user to pick from the store the specific product
that matches his requirement. In this case the segmentation didn‟t
go as expected (as seen in figure11) to the low amount of the
Figure 8: Store plan with scanner position. points that are exported from the MatterPak™ files which did not
contain enough points.
4.1 3D scanning
In this project we used “Matterport pro2” (shown in figure9)
which is a High-quality 3D capture camera with unlimited 4K
print quality photography. It helped in gaining professional photo
resolution output (134 megapixels) and 3D accuracy. It is also
recommended at scanning homes, apartments, hotels and
commercial buildings.

Due to the high technology used in this version, the scan has a Figure 11: Automatic segmentation for “Tradeline” store.
high 3D sensing using infrared; 20 seconds capture time per
scan/sweep, 99% accurate within range, Automatic 3D data As a plan B, manual segmentation was the way to extract a
registration, depth resolution equivalent to10 points per degree product from the mesh cloud output, of course it was not the best
(3600 points at equator, 1800 points at meridian, about 4 million solution to extract each and every product but it gave quite well
points per pano). output for an experiment. Using CloudCompare tools to manually
separate the required points and export them as a separate point
cloud that will be independently selectable in the application (see
figure 12).

Figure 9: Matterport pro2 3D scanner. Figure 12: Connected mesh and manual segmentation.
4.2.2 Avatar in real life. After animating and specifying the different
After exporting, filtering and the segmentation process, creating specifications, user can compare the selected product to different
an avatar that represent the user‟s body in the virtual world. Using similar products proposed by the application based on categories
“Unity” we created an avatar that allows the user to walk, watch and other dimensions. After specifying the product detail, user can
and rotate inside the environment and also a ray cast that helps the add the product to his virtual cart imbedded inside the application
avatar pick objects and buttons in the scene. Using “C#” coding which memories up to 20 products and can be modified at the end
language to create scripts that control the avatar and facilitate the of the shopping experience. Every product is also connected to a
connection between the user and the virtual avatar that represent URL that open the specific website page of the product to present
the user‟s body in the virtual reality world. Also using „Unity‟ to all the specification and also present the price and warranty.
create the interface and the buttons to allow the user to pick his
character (gender, color, etc..) to make the application more
immersive and to involve the user as much as possible in the
virtual world.
4.2.3 Product manipulating
Using the manual segmentation output to allow the user to
select each product separately and have a closer look to each and
every detail in the product to help the user choose with more
details and giving the ability to compare and test product using the
technology of VR. A separate mesh is created over each product
that allows the user to highlight an object (seen in figure13) when Figure 14: Similar products and specifications.
hovering over it, which will facilitate the selection of the required
product and limits the factor of low accuracy by users who are not 4.2.5 Customer support
very experienced in using new technology special the virtual The idea of integrating a virtual shop assistant that will be
reality headset. Adding animation that helps the user move around created as an avatar that exist inside the store and that is
the place to check the products in a close eye, which is technically represented by a 3D model that works with a scripted AI form that
changing the position of the camera and the specification of the allows the avatar to be connected to a data base of the most asked
camera to get the best quality out of the product and allows the questions and with a voice recognizing technology that allows the
user to rotate, zoom and manipulate the product as if it was user to ask verbally the question next to the avatar responsible of
between his hand in real life. And this fulfill the users physical the store (shown in figure15) and based on the data based already
need in shopping that most studies agreed that physical touch and built in inside the avatar, it will give the user the required answer.
freedom of manipulation is the most needed aspect in If the question or the answer was not clear there will always be a
really costumer service agent that is responsible to answer the
voice calls of users that are virtually in the store. With a click of a
button the user can contact the customer service throw the
headset‟s microphone to ask any question and receive the help
required to facilitate the virtual e-commerce experience. Also
giving the feeling of the interaction with real humans will help
immersing the user in the virtual world.

Figure 13: Highlighted segmented mesh.

E-commerce and which is not achieved in the 2D interfaced

platform that cannot allow a 3D model to be manipulated with the
user hands. The application also implement virtual buttons that
allow the user to connect to all the social media accounts of the
store he is visiting and allowing the user to check the stores Figure 15: Virtual assistant avatar.
website and also switch to the 2D platform if needed. 4.2.6 Payment and shipping
4.2.4 Product selection Each selected product will be assigned to an updated value
After adding the highlighted mesh over every product, scripting already saved in an external spreadsheet made by the store, and at
the ability to connect each product to allow a specific the end of the experience the user will be able to proceed to the
modification for each product based on different specification payment and the delivery process inside the app. Paying virtually
(color, size, etc..) and a small animation that represent the product and assigning the accurate address to deliver the shipment. User
individually and allows the user to manipulate the product to have will also be connected to with the shipping company to clarify any
a closer look and to test the product as if it was between his hands details and secure the order shipment to the write address.
 Acknowledgments
This research was partially supported by [M. RUDING LOU,
Researcher, Assistant professor, Arts et Métiers, France]. I would
like to thank my colleagues from [Megalodon360, Cairo, Egypt]
who provided insight and expertise that greatly assisted the
research, although they may not agree with all of the
interpretations of this paper.

REFERENCES
Figure 16: “Add to cart” window and pricing details. [1] How the Metaverse Is Reshaping. (2022, may 11).
Retrieved from datahawk:
5 EXPERIMENTATION AND EFFECTIVENESS https://datahawk.co/blog/amazon-and-walmart-
metaverse
By following the number of users that virtually visited the store
and how also following the time spent inside the store, a huge
[2] V., C. (2019, August 8). 3d natives. Retrieved from
numbers where noticed and a bigger time average was noted
https://www.3dnatives.com/en/laser-3d-scanner-vs-
compared to 2D E-commerce platforms. Results showing that the
structured-light-3d-scanner-080820194/
virtual E-commerce experience is a very attractive way for users
to explorer new places and the 3D scanner making it more
[3] Cao, B. (2017, april 29). digital initiative . Retrieved
immersive and attract more users to watch and buy from stores
from https://digital.hbs.edu/platform-
that use this technology.
digit/submission/alibabas-vr-shopping-fad-or-future/
Although the application was not totally finalized and the
[4] 3D Scanning With Structured Light. (n.d.). Retrieved
application was shared locally for surveying purposes, also
from https://bitfab.io/blog/3d-structured-light-scanning/
including a PC version to reach the maximum amount of users to
test the application effectiveness. Also there will be a survey for
[5] Needle, D. (2021, dec 29). Part of: What the metaverse
users to test the application from different aspects, such as the
means for IT pros, enterprises. Retrieved from
friendliness of the application, the ease of use and the quality of
https://www.techtarget.com/whatis/feature/The-
the visualization and experience; this will help in getting the best
metaverse-explained-Everything-you-need-to-know
out of the experience.

6 CONCLUSION AND PERSPECTIVES

[6] Needle, D. (n.d.). whalts.com. Retrieved from
https://www.techtarget.com/whatis/feature/The-
To conclude, virtual-commerce is the future of online trading metaverse-explained-Everything-you-need-to-know
and electronic shopping. This technology will take place all other
king of shopping and will connect the world together so that all [7] E. Grilli, F. M. (2017). POINT CLOUDS
humans can access shops and trade all type of products SEGMENTATION AND CLASSIFICATION
worldwide without paying for transportation or taking a lot of ALGORITHMS . Nafplio, Greece: The International
time or taking any risk to get certain products. The proposed Archives of the Photogrammetry.
application is allowing the user to walk freely in a 3 dimensional
spaces already scanned with a 3D scanner and using the VR [8] Matterport pro2. (n.d.). Retrieved from
headset, user can be immersed inside this virtual store to watch https://matterport.com/cameras/pro2-3D-camera
and buy all types of products, starting from fashion passing by
furniture ending with all types of technology. [9] Xu, S., Vosselman, G. and Oude Elberink, S., 2014.
Multipleentity based classification of airborne laser
The application can still be improved by adding and integrating scanning data in urban areas. ISPRS Journal of
more of the artificial and also training the application to fill the Photogrammetry and Remote Sensing, Vol. 88, pp. 1-15
data base and help it build a more smart and responsive
intelligence. Using the user friendly interface, the application is [10] MacQueen, J. et al., 1967. Some methods for
built in a way that allows the developer to add more stores with no classification and analysis of multivariate observations.
limits and also allowing the store owner to always upgrade and Proc. 5th Berkeley symposium on mathematical
modifying the store without the need to redo all the work but statistics and probability.
simply by adding more separated meshes that fulfill the new
version of the store. Also in the future, the application is aiming to [11] Sappa, A. D., & Devy, M. , 2001. Fast range image
be user friendly in a way that will allow basic smartphone users to segmentation by an edge detection strategy. Proc. IEEE
scan using their smartphone camera which will generate a 3D 3rd 3-D Digital Imaging and Modeling, pp. 292-299
model in a .obj format that can be used in the 3D environment
which will help in updating and improving the virtual [12] Vieira, M., & Shimada, K., 2005. Surface mesh
environment without the need of redoing all the work from scratch segmentation and smooth surface extraction through
and will also prevent from duplicating the work with a smaller region growing. Computer aided geometric design, Vol.
cost and more fun technical solution that will help the application 22(8), pp. 771-792
improve in a high pace.