Augmented reality-based border management

Abstract

A proficient border management has typically been tied to its capability to support information structuring and to make exchanges from the distributed sources. The lack of a proper access interface to information at the right time and the right place to conduct various activities. Augmented reality (AR) has been proposed as an efficient interface in order to improve the efficiency and the effectiveness of activities in the real world. There has only been a limited amount of research that has evaluated the effectiveness and usability of AR in the border management domain. This research aims to evaluate the effectiveness of border management and AR system integration to enhance the activities’ efficiency through improving the information retrieval process. The system development steps were adopted to design, develop, and evaluate the border management AR (BM AR) system. The system contains three AR services that include pointing to border objects, showing the borderline, and locating border objects. The system also integrates the information from different resources in an interoperable way using GIS web services. The results revealed the effectiveness of using AR for border activities, which can reduce the operating costs and effectively, access the required information for doing different activities in the border field.

1 Introduction

A border is an official line between two countries (Guo 2015; MD 2013) that is determined by the agreement of neighboring countries and determines the spatial range of activities both within and between countries (Brenner et al. 2008). As a border does not naturally exist on the ground, the lack of proper management results in many struggles between neighboring countries. In some cases, the activity of a country may exceed its boundaries by mistake, and it may enter the territory of a neighboring country. This has made border management one of the most important issues for any country. Iran, which has a common border with 15 countries, is no exception. Border management requires the use of appropriate data and decision tools (Guo 2015). In this regard, spatial data, which includes the location of border gates, border lines, and remote sensing images, and attribute data, such as socioeconomic census are very important (Guo 2015; Lovelock and Boyd 2006). The geospatial information system (GIS) has many capabilities for storing, managing, processing, and displaying spatial and attribute data of border features, which include border rods, outposts, lines, and mountains. Some research that has been conducted into the use of GIS in border management include (Riecken et al. 2003; Portman 2007; Latre et al. 2005; Cao et al. 2007; Knörchen et al. 2015; Moreno-Sanchez et al. 2007; Nunes et al. 2015; Chen et al. 2013; Donaubauer et al. 2006).

As many border-related activities are conducted outdoors, timely and easy access to the up-to-date data and the services of border objects is essential in the border field for effective border management, decision making, and permanent monitoring as well as the development activities and the maintenance activities. However, spatial data and services that are provided by the existing GIS systems in border management are not fully based on the users’ needs. They also require interpretation, which is due to the large amount of information that is provided. On the other hand, the borders between countries represent a simplification of the complex geo-political, political, and social struggles (Anderson and O'dowd 1999). This makes the border important from various aspects, which include geographic, political, social, and cultural aspects (Guo 2015). In regards to these different border aspects, the different organizations and institutions actively participate and collaborate in border management (Lovelock and Boyd 2006; Bharosa et al. 2012; Hoffman et al. 2013). In Iran, twenty-two organizations, which include the Road Maintenance and Transportation Organization, the Ministry of Energy, the Ministry of Interior, the Geographic Organization, the Border Guard Command, and the Ministry of Agriculture Jihad, have a role in border management from different perspectives. Each of them separately gathers, stores, and uses the spatial data, the attribute data, and the spatial services based on their needs. Accessing the data of these different data sources is necessary for effective and comprehensive border management. Since these organizations use different data types, data formats, and software tools data sharing has many problems, and there are currently no organizational or technical processes to create an appropriate data infrastructure to access the border data from various data sources.

To improve the access and the utilization of information at the right place and the right time, augmented reality (AR) has been identified as a ubiquitous technology that can be used as an interface to the real-world (Azuma 1997; Sharif and Sadeghi-Niaraki 2017; Wang et al. 2016). AR improves the users’ perception of and the interaction with the real world, and it simplifies their lives without forcing them to change their behaviour by creating easy-to-use and intelligent services (Carmigniani et al. 2011). Recently, AR technology has been used in different applications, such as cultural heritage (Ramtohul and Khedo 2019; Park et al. 2016), emergency management (Ahn and Han 2012), navigation (Huang et al. 2012; Kim and Jun 2008), flood management (Haynes et al. 2018), building information management (Chu et al. 2018), environment planning and management (Pierdicca et al. 2016), and education and learning (Bower et al. 2014). However, research demonstrating AR effectiveness and usability in border management remains scant. In addition, there is a distinct lack of developments that provide access to border objects information from different resources.

Although there is a growing interest in developing AR-based systems for border management and border applications (Charisi et al. 2021; Brioso 2020; Auchter 2020), there is not enough research that not only enables countries to access the right information in the border field but also enables developers to develop spatial services based on the spatial data infrastructure that facilitates information sharing between different organizations. Charisi et al. (2021) introduced the ARESIBO project which is a mobile and wearable AR project for border security to create enriched situation awareness. This project has been joint 20 participants from 11 countries to provide the right information from multiple sources. Brioso (2020), to fulfill the problem of the ARESIBO project, designed and developed valid use cases and scenarios based on the Concept of Operations (CONOPS) analysis of the project. Auchter (2020) discussed how the AR technology can be utilized to visualize border migrant morality and to create a new way for migrant memorialization crossing the US-Mexico border. These studies have not designed and evaluated the use cases for managing border activities, such as border object placement, in detail and have not considered the challenge of integrating different organizations in a country, which have different policies and duties in border management.

The objective of this paper is to develop a border management augmented reality (BM AR) system and evaluate the effectiveness of AR to enhance the activities’ efficiency. AR facilitates user-centric access to the required data and services by combining the real world and the virtual world. The system was designed by combining the ubiquitous GIS and AR technology to solve the border management problems in the border regions by providing the right spatial data and services of the border objects at the right place order to the user. The contexts of the users play important roles to access and integrate the information of the border objects from different organizations with different objectives. In this regard, GIS web services, such as WFS standard specifications are used to publish the spatial data and the services. This makes an interoperable system to access the comprehensive information of the border objects, which include border rods, outposts, and lines, from different sources. Therefore, we designed the AR-based border management services between various organizations in a country based on spatial data infrastructure and provided use case diagrams. We designed an experiment to evaluate the effectiveness of the BM AR system and compare it with the previous methods in Iran for the first time. The evaluation results showed the better performance of the BM AR system compared the conventional method.

In order to address this research issue, the structure of this paper is as follows. Section 2 is about the related works in AR. Section 3 describes the research approach of the paper that includes the designs and the development of the BM AR system. Section 4 describes the tests and the evaluations of the BM AR system. Section 5 contains the results and the discussions about evaluating the performance and the usability of the BM AR system. Finally, Sect. 6 concludes the paper and suggests the possible future work.

2 Augmented reality evaluation

There are studies that developed and evaluated the AR technology in different outdoor applications. In the border domain, Carpenter et al. (2018) developed an AR device to identify each of the pluralities of people and to retrieve security information of the person at a security checkpoint, such as a national border or other secure entry checkpoints. They have not evaluated the potential of an AR application for different scenarios of managing border objects and border demarcations. Huang et al. (2012) developed a method for the empirical study of the effects of AR in knowledge acquisition in pedestrian navigation by comparing the effectiveness of mobile maps, AR, and voice navigation. Carozza et al. (2014) developed and evaluated an ARC3D program for building reconstruction in urban applications. Milosavljević et al. (2010) developed a GIS-based AR application in the field of urban planning and emergency management. Haynes et al. (2018) designed a mobile AR flood visualization prototype to visualize floods on-site in an outdoor environment, and they evaluated its potential to determine how the app would be received by the experts. Zhang et al. (2020) proposed an innovative 3D flood AR visualization method that is based on a terrain model. Pierdicca et al. (2016) developed and validated an application using AR and a standard data layer in order to control the linear buffer strips in an environmental domain. Pombo et al. (2019) designed and developed a prototype of a mobile AR game application to promote outdoor learning in an urban green park. Zsila et al. (2018) designed the motivation factors of the famous outdoor geo-located AR game application, which is Pokémon Go. Mirauda et al. (2017) developed a prototype AR application to monitor water resources. Lee et al. (2015) developed an AR mobile application to search for and display boreholes in engineering. Meža et al. (2015) examined the potential of AR technology in civil engineering, especially in construction sites. Winzer et al. (2017) developed and evaluated a location-based AR for interactive storytelling in order to achieve a feeling of the presence of the events and the actions that are historically relevant to cultural heritage places. Park et al. (2016) proposed an all-in-one mobile outdoor augmented reality (AR) framework for a cultural heritage site. Ramtohul and Khedo (2019) presented a real-time location-based mobile AR system for cultural heritage sites. Litvak and Kuflik (2020) proposed an AR-based tourist guide system to enhance the cultural heritage experiences.

Based on the (Billinghurst et al. 2015; Dünser and Billinghurst 2011), there are five main evaluation methods for the user evaluation in the AR applications that include objective, subjective, qualitative, usability, and informal methods. Park et al. (2016) conducted an objective evaluation by conducting a quantitative experiment in order to measure the accuracy of the generated data and the speed of the camera tracking of a developed AR application. Duguleana and Voinea (2018) designed an objective and a subjective analysis by measuring the task completion time and a HARUS questionnaire in order to evaluate the cultural heritage AR applications for outdoor museums. Mirauda et al. (2017) evaluated AR technology using a subjective method through ordinary surveys and a questionnaire analysis with simulated emergency situations. Pombo et al. (2019) evaluated an outdoor AR game application using objective and subjective methods by analyzing the game completion time and a questionnaire to detect usability, acceptance, and the learning promotion of 74 students. Zhang et al. (2020) evaluated the effectiveness of the prototype of a 3D flood AR using two experiments through an objective method, questionnaires, and statistical methods. Ramtohul and Khedo (2019) evaluated an AR application for cultural heritage sites using a qualitative method and formal user observations in order to find its strengths and weaknesses. Champney et al. (2015) evaluated AR technology with the training of military tasks using the usability method for the simulation-based training capabilities in an outdoor field environment. Zsila et al. (2018) conducted a subjective analysis using an online gaming questionnaire in order to assess the motivation factors of Pokémon Go. Javornik et al. (2019) evaluated the effectiveness of AR content types using a subjective method, which was a questionnaire, in order to enhance the discovery experience and the learning experience of a university campus. Hartmann and Vogel (2018) evaluated a spatial AR-based mobile phone pointing application using the task completion time, which included the time to select the start target and the manipulated measurement target as the movement time, based on an objective evaluation method. Uchida et al. (2017) applied objective metrics that included the reaction time, the distance a vehicle is stopped from a pedestrian, and the user responses to evaluate the JARI-ARV (Japan Automobile Research Institute-Augmented Reality Vehicle) application in two scenarios. Plopski et al. (2018) designed a questionnaire to evaluate an AR-based Pressing Evaluation Training System. Ahmed et al. (2018) evaluated the user satisfaction with and the usability of an AR Furniture App using a questionnaire.

Among the AR evaluation methods, the questionnaires and the interviews are the most suitable methods to evaluate an AR system evaluation based on (Meža et al. 2015). Some research applied the existing questionnaires in order to evaluate the potential of an AR with a specific application. In addition, there are questionnaires that were designed or adapted to evaluate an AR potential for a specific application in the previous studies. Haynes et al. (2018 adapted a questionnaire to evaluate an AR with flood visualization. Chu et al. (2018) utilized the questions to investigate the AR potential to build information modeling, Zsila et al. (2018) extended a questionnaire to evaluate the AR-based outdoor games. Pombo et al. (2019) provided questions to evaluate an AR-based outdoor learning tool. Zhang et al. (2020) designed a questionnaire based on the principal of the Post-Study System Usability Questionnaire for flood visualization using 3D terrain models. Meža et al. (2015) utilized specific questions to evaluate the AR potential with civil engineering. Javornik et al. (2019) adapted a questionnaire to evaluate an AR regarding the discovering and the learning experience of an outdoor environment. Litvak and Kuflik (2020) designed a specific questionnaire to evaluate the user experience with tour guides at a cultural heritage site in addition to a system usability scale questionnaire.

The great potential of an AR was evaluated in many application areas in the previous studies, but it was not evaluated to solve the challenges of border management domains. Border management is a complex task and needs to access precise and updated spatial and attribute information, especially for conducting activities in border fields (Guo 2015). To manage and monitor borders, previous studies applied various spatial technologies (Nunes et al. 2015), such as web mapping (Knörchen et al. 2015; Moreno-Sanchez et al. 2007; Donaubauer et al. 2006), geo-portals (Riecken et al. 2003; Latre et al. 2005), and mobile mapping. However, the potential of AR technology as an emerging technology has not investigated how it can be used to help enhance the border activities. To fill the gap, this study designs and develops an AR-based border management system and evaluates its efficiency to enhance the border activities in the border fields. In this research, the questionnaire, which includes statistical and interview methods provided by the previous research (Chu et al. 2018; Haynes et al. 2018), will be applied in an experimental study. These methods will be adapted to the BM application. Therefore, the research mainly deals with BM-specific issues, which include the integration of BM and AR, rather than innovation of AR technology.

3 Design and development of the BM AR system

In order to evaluate the efficiency of an AR in order to enhance the activities related to the borders in the field, a border management AR system is designed and developed in this section. For this reason, the development steps, which include the research process, are adapted based on (Ananda et al. 2016; Geihs et al. 2012), which consider a user-centric design (Masoumi et al. 2021). Figure 1 shows the main five steps of the design and developing a BM AR system, which includes the problem definition, the requirement analysis, the conceptual design, the development, and the evaluation. By using the development steps to acquire the research knowledge and to understand the problem area, the main items of a BM AR system are designed and applied, which can be further developed.

Fig. 1

Development steps: the research process

3.1 Problem definition

The first step is the definition of the comprehensive border management problem. The current BM methods and systems are currently underutilized to manage the activities conducted in the border areas by different teams, and they are not particularly useful in Iran. Some of the problems with the current BM systems are the challenges with extracting the required information of the border objects, which include rods, outposts and lines, and accessing and sharing the required spatial and attribute information of the border objects in the border field for the different activities, such as monitoring and construction.

The current BM systems require spatial knowledge and efforts to extract and integrate the required information effectively from 2D maps or web\mobile systems. This can be time-consuming, because workers consume a lot of time to find their location and the related information, and it can create problems, such as the constructing a road in a neighboring country due to human mistakes. The maps generally cannot display large volumes of information of a border object, which is due to their cartographic limitations.

Only the users familiar with a region could identify the border objects, and then some of their information could be separately retrieved from the organizations based on their code. In regards to different border aspects, different organizations and institutions separately gather, store, and use spatial information and attributes as well as spatial services. This has led to the duplication of activities and heavy spending on border management. In this situation, access to border data was possible only within each individual organization. This means that there was no data sharing between the organizations, and it was only possible to access the border data by going to the organization involved, such as the Ministry of Energy.

In addition, border demarcation is an important task for the border management. Border demarcation is a field operation that accurately represents the location of the borders using the physical marks (Kagawa 2013). Jones (1943) discussed the methods of border demarcation with a line, the natural features, the markers, and the turning points. Surveying and border demarcation have faced many challenges, because the borders of countries are located in different geographic regions that include mountains, water resources, rivers, deserts, and forests (Jones 1943). Therefore, locating and visualizing the border objects, which include the border line and rod, are challengeable tasks in the border fields.

As specified by the problems with the current existing BM methods and systems, it is necessary to develop a system to integrate the border information from different data sources and access the required information of border objects in the field. Therefore, the objective is to design an AR-based BM system in order to provide the spatial data and the services to the user for border management in the border field.

3.2 The requirement analysis

The second step is the requirement analysis, which is conducted to achieve the objective outlined in the previous step to solve the current BM problems. According to the problem definition, it is necessary to integrate the data source of the organizations by maintaining the data ownership for each organization. For example, the rod number, and the next rod, and the previous rod distance are stored by the geographic organization and the region name. The protocol map of the rods is stored by the border guard command, which should be integrated to monitor the border rods. Three important scenarios are considered for the design and the development of a BM AR system are listed below.

• 1. In order to monitor the border, it is necessary to have access to the spatial data and the attributed data of the border features, which include the rods and the outposts.

• 2. The border features may be damaged or destroyed, so it becomes necessary to reconstruct them. Sometimes, a new rod or a new outpost may be required to be built. In these cases, it is required to locate the objects and their real coordinates.

• 3. The border lines that define the border between two countries are very important, because the reason for some of their struggles is the determination of the exact location of these lines on the ground. Hence, knowing the exact location of the border lines is always required in order to determine the geographic boundary of the activities of each of the countries to ensure that they do not encroach into the neighbouring countries.

The organizations and the institutions that are related to the border are the main users of the system. In other words, the system is designed so that only the members of each organization can use it. For each of these organizations, it is necessary to define the access level so that they can edit their organization’s data source, but they can only observe the data of other organizations.

3.3 Conceptual design

Based on the requirements, the design of the border management system is described in this section. Three AR services were designed in the BM AR system for each scenario, which included pointing to the border objects, locating the border objects, and showing the border line (Fig. 2). For these services, the spatial data and the attribute data are obtained with the integration of the Ministry of Interior, the Geographic Organization, and the Border Guard Command data sources using GIS Web service technologies in an interoperable way. The Ministry of Interior is responsible for organizing the working border groups in order to enhance the border security, which includes the country’s official entrances and exits, and the border residents’ trade exchanges based on the laws and the regulations as well as making decisions for the border market locations. Geographic organization manages the border surveying, demarcation, and the geospatial information of the border elements, such as the border lines, rods, gateways, and outposts. The Border Guard Command controls the land and the sea borders physically, maintains the border rods, and outposts, and it organizes the affairs of the country's border residents. It should be noted that these data sources are considered for the prototype system. Figure 2 shows the use case diagram and the main components of the BM AR system.

Fig. 2

Use case of the BM AR system

A BM AR system is designed based on the ubiquitous GIS (UGIS), which is beyond the previous generations of GIS systems, which include paper maps, digital maps, web GIS, and mobile GIS, to provide ease-of-use and context-aware spatial data and services (Kim and Jang 2012; Hong 2008). In fact, the ubiquitous term in UGIS expands the approach of any place, any time, and any user associated with the location-based services into the approach of the right place, the right time, and the right user (Reuter and Zipf 2008). The previous generations of the GIS information of the border objects are modeled in a virtual space (computers), but the modeling of the border objects have changed so that the computer-modeled data of the border objects are combined with the real world in UGIS, which is described in (Hugues et al. 2011). The users of the previous generation of GIS only access and observe border objects’ information in the virtual space, which makes different challenges to match the information with the real world. Therefore, UGIS makes it possible to use GIS functionality in the right place and at the right time, and it provides the added-value to the spatial data and services. AR is a UGIS technology that is applied in this research in order to design and develop a UGIS system to solve the problems of the border management, which is included in the problem definition step. A BM AR system provides context-aware services to access the right information of the border objects in the right place in the border regions.

By designing a BM AR system, by combining AR technology and UGIS, the main elements and their structure are modeled, which include users, contexts, border objects, and processes. Users of a BM AR system are the members of the organizations related to the border management, such as officials and experts. For each user, the demographic profile information, such as names, genders, and ages as well as organization names and the job position of the user in the organization are stored. In addition, the information of the data of the border objects is produced, and it is required by the organization of the user. In this research, three organizations are considered as the main users, which includes the Ministry of Interior, the Geographic Organization, and the Border Guard Command. The context information of the user is another element of a BM AR system that is used to provide the relevant spatial data and services to the user. The user location, orientation, and profile are the main contexts of the system. This means that the information, which include texts and 3D models, are only augmented to the real world when the user is in the right location and the right orientation and has the right profile contexts. Border objects, border rods, outposts, and lines are modeled using the spatial data and the attributes. The data of the border objects are retrieved from different organizations. Tables 2 and 3 show the spatial data and the attributes of the border roads, outputs, and their providers. Because the organizations that design different systems to manage the information of the border objects, which include using different databases, data formats, software, and structures. The GIS web services are used to publish the border objects’ data so that the organizations’ systems can communicate or interoperate with each other. Finally, the data integration is the data integration process that organizes the multiple data provider services for the different border management services. This process builds up the requests in a proper format to access the border object data and integrates them from the organizations as the provider services and parses the response. The data integration process is responsible to send a well-formatted request, which is in an XML format, to the organizations for any request from the client application, a BM AR system, and to extract the required data for each response from the organization. After the data of the border objects are integrated, they are prepared in a suitable format to augment to the border region. Figure 3 shows the BM AR system conceptualization.

Fig. 3

BM AR system conceptualization

Pointing to the border objects service involves two types, which include pointing to a border rod and pointing to a border outpost. Table 1 illustrates pointing to a border rod service, and Table 2 illustrates pointing to a border outpost service, which is both displayed in the real world. In this service, when the border rod/outpost are detected, the WFS web services related to the border object are retrieved from the different sources in the GML format. The information is then integrated in order to create the comprehensive information of the border object. After that, the integrated information is correctly rendered and presented in the camera frame as a web page. This service allows user interaction in the AR environment, which means that some of the border object information is augmented in the camera, and the user can view more information by clicking on it.

Table 1 The Rod data and their provider

Table 2 The outpost data and their provider

The locating of a border rod and an outpost service displays a virtual 3D rod and an outpost model in the exact place that is given through the user’s smartphone in the real world. The service augments the 3D objects stored in the smartphone’s database precisely at a given place through the smartphone’s camera. By showing the border line service, the border lines generated in the GIS environments are displayed through the user’s smartphone in a real-world environment. The border lines are augmented based on the user’s position and the orientation obtained by the tracking images. Hence, if the border line is located in the camera’s field-of-view, a virtual line is rendered on the camera frame.

The BM AR system is designed for organizations, which can be used by officials, experts, and other members that have access level permissions. The data access level permission and the type of service permitted on the data item being requested are applied in a BM AR system. The organization members, who are the users of the system, are assigned a separate access level permission for different services in the system by creating an account. All the organizations can observe the information from the different resources, but they can only edit their own data. The access level does not allow to display of information to the public users.

In addition, the system is designed to have services to edit data. As the data is gathered from different sources of the three organizations, each organization member can only edit their border data source through a web service. For example, members of the Ministry of Interior can only edit the facilities and the name of the closest residential area (Table 1) of an outpost. They cannot edit other information for that outpost. The web services make a secure data management infrastructure by maintaining the data ownership (Bertino et al. 2008). In addition to maintaining the data ownership by each organization, the advantage of a web service is that if an organization changes its data, these changes are applied automatically and immediately, and other organizations can see these changes (Jelokhani-Niaraki et al. 2018).

In this research, natural feature tracking (NFT) was used to create the mobile augmentations for the evaluation. NFT is a markerless technology and an image-based tracking mechanism to incorporate and interact with as much of the real world as possible, which only uses the natural features that are easily detectable in a scene, such as edges or corners (Haynes et al. 2018; Gupta et al. 2018). The scene to be detected must be rich in details and have a preferably high contrast (Blanco-Pons et al. 2019) so that the natural features that exist in the environment are tracked using a tracing image database of the scene. Actually, the tracing image database is used to estimate the position of the features and render contents through the camera of the smart device relative to the user's position (Haynes et al. 2018).

3.4 Development

In this step, a BM AR system is implemented using the selected software, hardware, and programming language. By doing this, the existing AR development kits/tools were considered to integrate an AR system and the border management. By utilizing and adapting the existing AR development kits/tools, there is no need to develop the core functionality of the software/hardware to enable the use of AR in the border management. This allows a greater focus on the evaluation of a BM AR system to address the problems that practitioners are confronted with in the BM. The Wikitude development kit is used in this research, because its tracking method is based on the NFT, and its development kit is easy-to-use for developers. The Wikitude’s NFT method can track objects up to 20 m away, which is based on (Amin and Govilkar 2015), and it is an advantage for border object tracking due to the security issues. Wikitude, which is available for free as well as commercial SDK, provides a cross-platform AR for multiple devices and operating systems because of the web technologies that use it, which include HTML, JavaScript, and CSS. Based on (Jooste et al. 2015), Wikitude supports the widest platforms compared to other AR SDKs, which include ARLab, iPhone ARToolkit, DroidAR, Layar, Metaio, and PanicAR. It supports Android and iOS, Google Glasses, Blackberry, and Windows Mobile platforms. All of Wikitude’s developmental features are available through an open-source software development kit (SDK) for AR development that allows for full customization of its source code (Chu et al. 2018). This enables scene recognition and geo-tracking for developers to recognize and augment entire real-world scenarios with virtual content.

The virtual 3D model of the border rod and the outpost were created using Agisoft software in the .obj format. To use the model in Wikitude SDK, the.obj model was converted to a WT3 format using a Wikitude 3D encoder. The mobile platform was used to implement and evaluate the BM AR system in this research. The system can be used on Windows, Android, and iOS mobile devices, because the use of Wikitude and web services has made it available cross-platform.

The prototype system was developed to evaluate the Bazargan region, which is a border region between Iran and Turkey. Figure 4 illustrates the map of the Bazargan region. A replica of a part of the region was created to accomplish this, which included its border rods, outposts, mountains, and gateways. The spatial data and the attribute data of the rods, outposts, and lines, which include the GIS layer, were collected from different organizations. In order to implement the system, a PostGIS database, a geo-server, and an Android smartphone were used.

Fig. 4

The Bazargan region: the border between Iran and Turkey

Figure 5 shows the overall structure and the architecture of the prototype system. The required border spatial data and the attribute data are supplied by the three organizations. For each organization, there is a database where their data is stored. In this layer, some data management tasks are performed that include data retrieval and updating. Over this data sources layer, there is a web service layer, which provides access to the data sources through standard GIS web services. The map server works as an application layer, and it retrieves the required information according to the client requested from the GIS web services. It then integrates them to send to the client. The mobile application layer, which is the client layer, consists of a mobile AR application in order to provide the AR services to the users. This layer consists of the AR interface where the integrated information or the virtual 3D model, which is retrieved from the device database, is rendered in the user’s camera.

Fig. 5

The BM AR system architecture

To implement NFT tracking, the images of the study area were collected, which included the images of real features such as the border outpost, rods, gate, and mountain. The images and their descriptors were stored in the Wikitude. When a user runs the AR part of the BM AR system, each frame of AR video is analyzed to get the camera position. This position is used to retrieve the right information from databases. For retrieving information, the request sends to each organization server to access its WFS service of the spatial information around the position. The received spatial data, which were sent back in GML format from the geo-server of each organization, are integrated to create the right information of the border object in the frame. Figure 6 and Algorithm 1 shows how the BM AR system visualizes the virtual information of the border objects.

Fig. 6

The sequence diagram of the BM AR system

4 Testing and evaluation

An experimental study was designed in order to simulate three scenarios to evaluate a BM AR system to facilitate conducting activities in the border field. According to the problem definitions in the border management, a BM AR system was proposed in this research to integrate AR and BM systems by providing the comprehensive required information in the right place in the border field. It is expected that a BM AR system will increase the performance of the border activities, which include the location of the border rode and line, and increase the competence to conduct the border activities because it provides a simpler and more efficient tool to access the required information of the border objects for any users, which includes users who are not familiar with the border region and do not have spatial knowledge, and to enhance access to the required information of the border objects by integrating the different data sources.

In the experimental study, three main use cases are covered by the BM AR system, which is described below.

• What is the information of the border objects? The participants were asked to find the rod number, the rod protocol map, and the rod material for the border rod object, as well as to find the outpost name, the outpost facilities, and the number of border signs. This scenario was designed to evaluate accessing the integrated information from different sources by pointing to the border rod through the system.

• Where is the border rod located? The participants were asked to find the border rod, which had been destroyed on the ground, and mark its location. This scenario was designed to evaluate locating the service of the border objects by visualizing the 3D virtual rod when the participant is near a predefined location.

• Is the border rod located in our region? The participants were asked to find the country where the border rod is located, which was either Iran or Turkey. This scenario was designed in order to evaluate whether the border line service facilitates the border line demarcation, conducts activities in the border regions, and solves the countries’ disputes by visualizing the border line between the two countries on the ground as a result.

To conduct the three tasks, the participants needed to conduct the following steps. First, they needed to create an account. After that, they needed to configure a configuration of the user profile, and then they needed to select the desired service from the screen that displayed a list of the available border management services.

To conduct the experimental study, the population group contained 32 experts whose organizations included the Road Maintenance and Transportation Organization, the Ministry of Energy, the Ministry of Interior, the Geographic Organization, the Border Guard Command, and an academic university. This experimental evaluation was conducted in four steps that included (i) a PowerPoint presentation that was shown to introduce a BM AR system’s capabilities and to use cases in the study area for all the participants, (ii) the system was examined by each participant in order to perform the use cases covered by the system, (iii) the participants were asked to fill in a questionnaire to reflect their feedback with rating different aspects of the system performance, and lastly (iv) interviews were conducted with the participants. Two devices of Samsung Galaxy Note II were used in the user study. Post-Study System Usability Questionnaire (PSSUQ) (Lewis 1992) was used in this research to measure participant feedback to the BM AR application (Table 3). To complete the questionnaire, the participants were asked to rate each question of the feedback of the BM AR system on a scale that ranged from 1 to 7, which indicates strongly agree and strongly disagree, respectively.

Table 3 The questionnaire

5 Results and discussion

5.1 Overview of the BM AR system

Some snapshots of the three services in the modeled Bazargan are shown in Figs. 7, 8, 9 and 10. Figure 7 shows snapshots of the border rod and outpost pointing. When a user points to the border rode and outpost, the GML data were retrieved and integrated from the three organizations using the WFS service, after the camera’s position calculation. The border rod information was retrieved from Geographic Organization and Border Guard Command as shown in Table 1. According to Table 2, the outpost data were retrieved from Border Guard Command, Geographic Organization, and Ministry of the interior. The three organizations have all the rod and outpost information that the user needed to conduct the first use case.

Fig. 7

BM AR system a border rod pointing, b border outpost pointing

Fig. 8

A snapshot of the locating the virtual border rod

Fig. 9

A snapshot of border line service

Fig. 10

Outdoor testing a border rod pointing, and b locating the virtual border rod

Figure 8 shows a snapshot of the locating the border rod service. The user needed this service to conduct the second use case. For running this service, a user inserted the position of the destroyed border rod. After that, when the user is located near the position, the 3D model of the border rod was retrieved from the mobile database. The 3D model was augmented at the position the user inserted.

Figure 9 displays a snapshot of the border line service. This service was required to do the third use case. In the border line service, the border line between Iran and Turkey was rendered based on the line GIS data that was retrieved from the Geographic Organization using a WFS service. The data was in a GML format that was possible to extract the point positions of the border line. After the line had been augmented, the user could recognize where the border rod is located, in Iran or Turkey.

Figure 10 also shows snapshots of the border rod pointing and locating the border rod services in an outdoor environment to test the potential of the BM AR system. In the border rod pointing, the protocol map was augmented on the border object. Figure 10b shows a virtual border was located at the predefined position in the real world using the AR-based border locating service. It should be noted that the system was developed in the Persian language due to a request from the Iranian organizations.

5.2 Characteristics and experiences of the participants

The participants were comprised of 27 males and 3 females. The participants were aged 25 to 60 years old so that the mean age was approximately 38 years. Among them, 24 of the participants’ had 5.5 years of work experience in border management, which ranged from 1 to 18 years. The participants were from different organizations in this study. Two of the participants are from the Road Maintenance and Transportation Organization, one participant is from the Ministry of Energy, four participants are from the Ministry of Interior, sixteen participants are from the Geographic Organization, three participants are from the Border Guard Command, and four participants are GIS students from a university. Among them, 20 participants have experience in border regions, which include surveying, construction, and flying. The medium for getting information about border objects is usually paper maps and documents, but sometimes GPS devices are used to find the precise location of the objects. Mobile devices, such as smartphones and tablets, have been recently used to search for information from digital maps and websites. Nine of the participants had experience with AR in different fields.

The participants also reported some problems with the current methods that they encountered in the border regions. Finding the exact location of the border objects, especially the border line and rods, is one of the problems that most of the participants noted. This problem created struggles between the two countries. For example, one of the participants expressed in his experience that they had built a building in the border region, and they found that part of the building was located in the neighboring country after the construction. Therefore, they were forced to demolish the building, which wasted time and money. Finding information about a border rod when they are in a border region is another important problem some participants stated. This work is very difficult and even impossible using maps and documents. This created problems to find out where they were exactly.

5.3 Evaluation and data analysis

The responses of the participants for question numbers 10–25 are statistically summarized in Table 4. The table for each question shows the mean (μ), the standard deviation (σ), the lower quartile, the middle quartile, and the upper quartile, which are denoted as Q1, Q2, and Q3, the inter-quartile range (IQR), the measure of skewness, and the measure of kurtosis. In addition, the box plots of these questions are shown in Fig. 11. The results show that the majority of the participants thought that the system was easy, simple and comfortable to use (Q.10, Q.11, Q.13), was easy-to-learn the system (Q.14), productive (Q.15), provided effective and clear information (Q.18, Q.19, Q.20), provided a clear and pleasant interface (Q.21, Q.22, Q.23), and contained the functions to complete quickly the tasks of the border management (Q.12), because there are positive skews and median ratings of around 2. The participants were overall satisfied with the BM AR system due to obtained values for the skews and the median rating for Q.25. Fixing the error messages (Q.16), recovering the possible mistakes (Q.17), including the required functions (Q.24) are three questions that are negatively skewed. However, the maximum value of the participant’s responses to these questions is 4, so it can be stated that most of the participants agreed with them. Q.12, Q.13, Q.14, Q.15, Q.19, Q.22, and Q.25 are questions that have a positive kurtosis and outliers. These questions have a small value of IQR, less than or equal to one. The positive kurtosis and the small value of IQR mean that the opinions for the participants were close together.

Table 4 Statistical measures of the questionnaire (question number 10–25)

Fig. 11

Boxplots of the questionnaire (question number 10–25)

Figure 12 shows the averages of the participant responses in the four groups of questions including system usefulness, information quality, interface quality, and overall quality. Based on (Lewis 1995), the average of questions Q.10 to Q.15 represents the system usefulness, the average of questions Q.16 to Q.21 represents the information quality, the average of questions Q.22 to Q.24 represents the interface quality, and the average of the question Q.25 represents the overall quality of the BM AR system. The lower score of the overall quality confirms the overall agreement of the participants on the quality of the BM AR system in the border activities, although they had some concerns in the other groups: system usefulness, information quality, and interface quality.

Fig. 12

The average of the participant responses in four groups of the questions

In addition, to calculate the centrality and the spread of the participant responses, the correlations were calculated between the participant responses to the meaningful and the relevant questions. In order to understand how much the responses of the participants in the feedback of the BM AR system are related to their experience in border management, which are familiar with AR, and conducting an activity in a border region, eight questions (5–7 and 10–25) were considered in the correlations calculations. The correlations were calculated using Spearman’s rank correlation coefficient, which is a nonparametric measure (Kumar and Abirami 2018) that is used to measure similar patterns (Levitt et al. 2019) and is useful for non-normal variables (Sabry et al. 2014). This coefficient was selected because it is suitable for ordinal data and is relatively robust to outliers (Schober et al. 2018). Table 5 shows the symmetric Spearman correlation coefficient matrix between all of the selected questions.

Table 5 Symmetric matrix of Spearman's rank correlation coefficients

Our findings in Table 5 show the positive and the negative correlations between the questions. Overall satisfaction with the BM AR system (Q.25) correlated with the experience of the participants in the border management and AR. The moderate negative correlation between Q.25 and Q.6 with a 98% confidence means that the participants who were not familiar with AR had a stronger satisfaction with the BM AR system for border activities. However, these positive correlations do not imply a lack of support from the participants familiar with AR, which the scatter diagram in Fig. 13 demonstrates (a). Figure 13a shows that the relationship between the AR experienced participants and the overall satisfaction of the participants is quite complicated, but the higher ratings suggest that the participants did find the system useful. The moderate negative correlation between Q.25 and Q.5 with a 96% confidence can be explained the same way as the participants who have more experience in border management satisfied strongly with the system. The productivity of the system (Q.15) was moderately and negatively correlated with the participants’ experience in AR (Q.6) with a 97% confidence, which the scatter diagram in Fig. 13 demonstrates (b). The moderate positive correlation between Q.15 and Q.7 with a 99% confidence means that the participants who conducted an activity in a border region had a stronger agreement with the productivity of the system for conducting border activities. These results could signal that the prototype system could attract the attention of the participants, especially the border specialists. In addition, the participants’ experiences in AR and in conducting border activities showed a moderate positive correlation with the easy-to-learn feature of the system (Q.14), which produced approximately 99% and 97% confidence, respectively. These results indicate that the proposed system is useful for border management, however, certain improvements are needed, especially in the interface and AR parts.

Fig. 13

Scatter diagram a between AR experience (Q.6) and the participant satisfaction with the BM AR system (Q.25) and b between the productivity of the system for the border activities (Q.15)

5.4 Comparison analysis with conventional methods

To conduct the comparison between the BM AR system and the conventional methods, the participants were asked to perform three use cases covered by the system with both conventional methods and the proposed AR system. The conventional methods to perform each task were conducted as follows:

• The existing method to find the information of the border object is the web GIS system. This system, developed by the Geographic Organization, contains the border information that are provided by this organization. To find other information regarding the border objects, the participants were required to search for the required information on the other organization's systems, e.g. the rod protocol map, outpost name and outpost' border signs from Border Guard Command, as well as outputs from the Ministry of Interior.

• The existing method to locate the border rod on the ground is the differential GPS method. With this method, the rod position was placed on the ground. To this end, the Trimble GNSS receiver was used.

• The existing methods to find where the border rod is located are differential GPS and surveying methods. To perform this task, the participants were needed to locate the border line on the ground. The differential GPS and surveying methods were used by the participants to place some points of the line on the ground. With the points, the participants drew a line on the ground.

In this step, we implemented paired t-test to analyze the efficiency of the BM AR system for comparing the time taken by the participants in performing the tasks. The time taken by the BM AR system to complete the task 1 (µ = 2.710, σ = 0.646), task 2 (µ = 2.627, σ = 0.860), and task 3 (µ = 2.553, σ = 1.162) was significantly lower than the time taken the complete the task 1 (µ = 5.143, σ = 0.703), task 2 (µ = 31.073, σ = 1.824), and task 3 (µ = 39.760 σ = 2.192) by the conventional methods. Table 6 shows that results of the paired samples t-test in time taken to perform three use cases. The results show that the null hypothesis (the time-mean difference between the BM AR system and the conventional methods is zero) cannot be accepted in the less than 5% level of confidence (p < 0.05). It can be concluded that the BM AR system is highly efficient in conducting border activities.

Table 6 Paired samples T-Test between the conventional methods and the BM AR system for time completion

After performing three tasks with the conventional method, the participants were asked to fill in the questionnaire. Figure 14 shows the box plots of the responses of the participants to the questionnaire for the conventional methods. The results reflect the better performance of the BM AR system compared to the conventional methods from the participants’ viewpoint. Table 7 shows the results of paired samples t-test of the participant responses between the BM AR system and the conventional methods in p < 0.05. The responses to the most questions were obtained higher for the BM AR system than the conventional methods, approximately 24% on average. The average of the system productivity (Q.15) was obtained for the BM AR system (µ = 2.40, σ = 1.303) with the maximum difference higher than the conventional methods (µ = 4.00, σ = 1.619). The difference indicates that participants are about 40% more willing to use the BM AR system than the conventional methods. Providing clear and organized information on the screen (Q.18 and Q.21) are two questions that achieved the higher mean value for the conventional methods (Q.18: µ = 2.67, σ = 1.124; Q.21: µ = 2.73, σ = 1.380) than the BM AR system (Q.18: µ = 2.70, σ = 1.664; Q.21: µ = 2.93, σ = 1.911), less than 0.2 difference. These differences are due to the fact that the conventional methods have been used for many years in various fields, and their user interface has been well-designed and improved. According to Table 7, the null hypothesis cannot be accepted in the questions of the questionnaire in the 5% level of confidence except for Q.16, Q. 17, Q.18, Q.21, and Q.24. The results of comparing the system with the conventional methods confirm the previous results of the system evaluation that the BM AR system is much easier, simpler, and productive as well as needs much less time to conduct border activities than the previous methods from the participants’ viewpoints. However, some improvements are needed, especially in the field of user interface and how information is augmented.

Fig. 14

Boxplots of the questionnaire for the conventional methods (question number 10–25)

Table 7 Paired samples T-Test between the conventional methods and the BM AR system for the usability questionnaire

6 Research advantages and limitations

The results of the evaluation and the interviews provided the advantages and the limitations of the proposed system. A comparison of the BM AR system with the conventional methods or systems indicates the effectiveness of the system (Table 8). It should be noted that the results of the comparison were obtained from interviews with evaluation group of participants who have already had experience working in the border field using conventional methods. The proposed system integrates the border data of the related organizations in an effective way so that when the data is changed by an organization, this data is updated automatically and immediately. In addition, a BM AR system provides access to the border data based on the location and the orientation of the user in the border field. This helps the users who are unfamiliar with a border region to identify the border objects and their locations on the ground. Access to the locational information and the attribute information of the border objects decreases the human mistakes and decreases the conflicts between the countries as a result. The system can display all the information of the object without any limitations, and the users who do not have GIS knowledge can easily use the system without the need for an interpretation of the information.

Table 8 Comparison between the BM AR system and conventional methods

It should be acknowledged that the research has limitations. Since NFT technology was used as the tracking method in the system and depends on detecting the natural features in a scene based on the captured target images, the radius of object detection is limited. This issue depends on the quality of the target images. Even though this radius was acceptable for the modeled Bazargan region, it is better to use sensor-based technologies for a better radius detection for the real regions. In addition, a BM AR system is limited to three scenarios, which include border rods, outposts, and line objects. Finally, it can be noted that the small size of the population group used for the evaluation was a problem, even though it is comparable to other studies, such as (Chu et al. 2018).

In addition to the above limitations, the concerns and the expectations of the participants that were obtained through the interviews in three parts of the system’s properties based on (Olsson et al. 2013) include (1) the system functionalities, which include some participants that were worried about the inefficiency of the system when there was no Internet connection. They believed that the system should be designed in a way that can work in this situation, because Internet access may not be available at any moment in the border region. Some participants were worried about using the system from far distances from the border objects, because they said that they could not go near the border rod in some cases, such as while flying or being bound by a contract. The data security management was one of the other expectations of most of the participants. In addition, access to precise locations and orientation information was another concern of most of the participants to augment information and virtual models. This is one of the challenges that many mobile GIS programs have. (2) Information Content, which includes displaying the required information for each organization. In other words, personalizing information was one of the topics of discussion of some of the participants. They did not believe that all border object information should be displayed to all users. Instead, useful information should be provided depending on the current conditions and the user. For example, the users from the Ministry of Energy should only be able to access the required information from different sources about the border objects and should not have access to other information. (3) Interaction with and presentations, which include displaying border information flexibly as well as simple interaction with the system was the other expectation of the participants that were raised in the interviews. In addition, it would be better to design the interface to be more user-friendly and simpler to interact with. In other words, the help information should be very clear.

7 Discussion and conclusion

The emergence of AR in recent years has led to the formation of new opportunities in different fields. AR creates an easy-to-use and intelligent environment with a combination of the virtual world and the real world, which is an environment where any person can have access to the right data and services without any constraints. In this paper, a BM AR system was designed and developed in order to evaluate the potential effectiveness and the usefulness of AR technology with border activities. The system not only provides integrated and interoperable border data management of different organization sources, but it also provides efficient monitoring, a performance improvement, and the maintenance of activities by accessing the appropriate data and by combining the real world and the virtual world. This system is designed to have three AR services, which include the pointing service, the locating service, and the line service. These services retrieve and integrate the spatial data and the attribute border data from the Ministry of Interior, the Geographic Organization, and the Border Guard Command data sources through a GIS web service and augment them according to the location and the orientation of the user.

The prototype system was implemented and evaluated for the modeled Bazargan border region between Iran and Turkey. The evaluation was conducted using 30 participants from different organizations. The evaluation results can be interpreted such that the participants who have experience in border management perceived the system as productive and easy-to-conduct activities in a border region. The agreement of the productivity of the BM AR system was approximately 80%. The BM AR system overall satisfied the participants with 90% agreement. However, the participant comments revealed that greater and complex scenarios were required to be tested for a serious application. The results of the comparison between the BM AR system and the conventional methods illustrated the efficiency of the BM AR system for conducting the border activities in the border region.

The results of the experiment study provide some support for the idea of the potential of a BM AR system in border management scenarios in Iran and also in other countries. The border monitoring and the demarcation are one of the challenges of different countries. Finding the points in the border region and locating the border point pyramids were some of the problems Kosovo and Macedonia experienced with respect to border management (Meha et al. 2010). Locating border object services of a BM AR system facilitates the problems that Kosovo and Macedonia experienced. Nepal and India had different disputes with border demarcation in the forest regions, cultivated land, and river courses (Kansakar 2012; Baral 2018). In these border regions where surveying and making border markers and lines on the ground is very difficult, using a BM AR system has the effective potential to visualize the border lines and markers without the need to destroy the ground.

The concern of the participants in data security of a BM AR system shows the importance of the unanswered questions from the ethical aspects of the AR technology to accept this technology in different domains, which include border management. Based on (Heimo et al. 2014), the ethical aspects, which include privacy, ownership, peer surveillance, and surveillance, should be taken into account with the acceptance of AR technology. As the organizations are the main users of BM AR systems, the access level method of the system is a step toward establishing the organization’s privacy and the security policies to control the access of the undesignated users and to protect the important information. In addition, using GIS web services enables the system to maintain the data ownership. However, other issues, such as peer-to-peer misuse, which include cyberbullying and hacking, remain unanswered with the designing and the evaluating of a BM AR system.

Therefore, while we believe that a BM AR system can be useful in border management, and further work should be conducted, such as testing the use cases using other criteria, testing other scenarios in border management, enhancing the AR tracking method, and finding methods to solve ethical issues. To evaluate the BM AR system, other criteria including economic, social, environmental, and accuracy could be compared with the conventional method for future work. To expand the scope of the paper, AR technology could be investigated in other scenarios of border activities, such as locating border watchtowers, which could have high coverage of the border region and with creating secure border maps. Developing and evaluating more precise possible tracking algorithms, e.g. integrating the NFT method with the Global Satellite navigation systems and applying RTK correction, which makes it possible to access information from far distances of border objects, are possible future research topics. Finally, research on finding solutions to solve ethical aspects of a BM AR system, which include the intended uses and the unintended uses regarding the surveillance and other private and governmental uses of the data, ownership, and peer surveillance is considered as another possible future work. For example, from the organizations’ privacy and security assurance, it is needed to create a module to render the information on the border objects through an AR camera. It can be conducted by adding a module to an AR camera that can detect where and when to stop showing the information due to a security reason or a permission reason.