Active robotic search for victims using ensemble deep learning techniques

The standardised environment has been made to achieve the requirements of a NIST orange environment [52]. NIST standards are used in Rescue Robotics completions and research to make possible comparisons of results.

Three levels are defined in these standards: yellow, orange and red, with the orange level being between the other two in terms of the complexity of the scenario. This level is characterised by a floor made of different materials, with ramps or stairs, which require a certain degree of agility in manoeuvring. In terms of planning, it is necessary for the robot to be able to fully map the environment, as it can be reconfigured in real-time, simulating collapses, opening or closing doors and blinds, etc. Due to the limitations of the robot's own lidar, and this work being a proof of concept, environments with more than one floor have never been addressed. However, a future line of interest may be to incorporate an advanced SLAM module into the system to manage several floors.

Two scenarios following orange standard were set up inside a test area of 49 m² (7.6 m × 6.5 m). They can be seen in figure 2. Both simulate a house: they have a little entrance with a door, behind which there is access to different rooms with some tables and other everyday objects and several pieces of debris on the floor, such as a semi-transparent pallets, walls, etc. Some human mannequins were placed in the role of victims. Not all the obstacles hide humans. This aims to study the system's behaviour with false positives and whether the exploration order is natural and efficient (e.g. if it looks first on the nearest obstacles).

Zoom In Zoom Out Reset image size

While the scenario in figure 2(a) aims to demonstrate the validity of the system and its algorithms in a generic post-disaster environment, the objective of the second is to recreate an environment with floors containing obstacles, ramps or other debris that make it difficult for a conventional robot, i.e. a wheeled robot, to pass.

The main floor of the CAR building, which consists of a series of offices and information technology (IT) laboratories, with ground plan dimensions of 20 m × 35 m, has been chosen as an open controlled environment. Figure 3 shows a blueprint of the facilities.

Zoom In Zoom Out Reset image size

Unlike the previous environment, this is not a place where damages as a result of natural disasters can be seen; however, it has been chosen since the system has been designed to work in dangerous situations, such as gas leakages where no material damages have been produced. The main floor tests also allow us to study ARTU-R's behaviour in much more crowded environments in which a vast variety of objects (cables, CPUs, robots, chairs, tables, showcases$\ldots$) are present and have to be avoided.

The first step in implementing the system was to train a detector to identify which type of room humans were being searched. As introduced in [31], this determination can help to improve success rates when doing indirect searching.

To perform this task, it was first necessary to search for a dataset that had such a sufficiently large number of images of the different rooms in which to carry out the exploration. It was found that [57], a house room dataset with 1158 images, met these requirements. No data augmentation was considered necessary. The data, however, have to be labelled.

There were five possible room types: bathroom, bedroom, dining room, kitchen and living room. At first, the possibility of having room types more closely linked to the professional sphere, such as offices, computer rooms, workshops or meeting rooms, was missing. However, although it was considered to combine the dataset with another one that had some of these elements, it was seen, as will be discussed later, that the results, with the reduced spectrum of options, timed out to be quite good.

For the identification of rooms, the YOLOv5 CNN [58] was used, in its small size, as it was considered a relatively simple task to tackle. Unlike other models, YOLO can initially place different bounding boxes over an object and decide on the last step, based on probability scores, which to take. In the same way, this last version of YOLO has been prepared to work with partially occluded objects. Figure 5 presents a conceptual approach to its architecture.

Zoom In Zoom Out Reset image size

As shown, the network is divided into three subnetworks. The first one, called Backbone, acts as a feature extractor. It combines convolutional and CSP layers, presented in [59]. The second step, called Neck, acts as a feature aggregator, using a spatial pyramid pooling and a path aggregation network. Finally, the Head processes that information and predicts the bounding boxes and the related probabilities, which are the net outputs.

For this case, the net was trained on 80% of the data and 20% of the image bank was reserved for testing. 20 epochs were set. To see how the system worked, the training process was divided into three stages: one of high approximation and two of refinement, in which a 96.99% mAP was achieved. a precision of 92.66% and a recall of 93.2%. Training curves are presented in figure 6. Although these results might indicate the existence of some overfitting, section 4.1 will demonstrate how the network can function adequately with images extracted from other contexts.

Zoom In Zoom Out Reset image size

A sample of some data from the training process can be seen in figure 7. The confusion matrix, displayed in figure 8, demonstrates that the accurate detection of rooms is achieved in a minimum of 88% of instances. Notably, the predominant sources of confusion arise from the misclassification of kitchens as dining rooms and vice versa, manifesting in 7% and 8% of cases, respectively. This phenomenon can be attributed to the presence of kitchens with warm colours and furnished with chairs and dining tables, as it can be observed in figure 9.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

With the exception of the dining room classification, false positives, i.e. instances where parts of the environment are incorrectly categorised as one of these room types, occur in fewer than 30% of cases. These outcomes can be considered acceptable. As elaborated in section 4.1, they will not be a problem during the integration of the system. This is because detections with probabilities below a specified threshold will not be taken into account. Consequently, false positives, similar to the corridor illustrated in figure 10, will either disappear or remain at a very low level.

Zoom In Zoom Out Reset image size

Once trained, when used, the model returns a bi-dimensional array with as many rows as objects detected and which contains, in the columns, the bounding box coordinates and the detection probability. As shown in algorithm 1, the useful information is inserted in a one-dimensional array, detRooms, which host, at element i, the probability that the room is of type i. The correspondence between vector indexes and room types is presented in table 3.

The next detector to be configured was the object detector. No training was done for this subsystem: the YOLO network itself has, from the moment of downloading, a file with weights already trained for the detection of different elements, such as persons, kitchen utensils, means of transport or everyday objects. As it has been done for the Room Detector, the results coming from the CNN have been processed and inserted into a similar vector, as the algorithm 2 shows.

Since the system's objective is detecting persons using IS, the problem posed here was 'what everyday objects and items could be correlated with a person' in a post-disaster environment. What, in some situations, seems obvious—the remote control is expected to be near a television—was not so in this case, as there are many activities that a person can be doing at any given time of the day.

Therefore, the following assumption was made: it was assumed that, after a disaster—such as an earthquake, for example—the people's position tends to be the same as the previous position before the disaster. Consequently, intelligence systems do not need to be trained with post-disaster data but only with everyday images. Even though this premise may at first glance be considered unrealistic, works such as [60, 61] point out that common-sense responses are largely overlooked due to the excitement produced by natural phenomena. Moreover, contrary to popular belief, good behaviour during a disaster does not necessarily mean a lot of moving [62]. In this line, New Zealand's government guides for acting before, during and after earthquakes [63] recommend that citizens not move far but stay where they are and adopt a drop-cover-hold position.

The chosen dataset for the task was Common Objects in Context (COCO), [64], developed by Microsoft with 330 000 images on various aspects of everyday life.

With it, a database relating to the presence of humans and objects was created. The two presented detectors were applied to each image, and the probability associated with each object or room was stored in the database. If any human was detected in the picture, a tag was added to that database entry. Using the terminology of algorithms 1 and 2, the inputs of the random forest are the vectors detRooms and detObjs and the output, the probability of appearance of a human in that region.

Algorithm 1. Rooms detection algorithm.
Function DetectRoom(picture) is
$RoomsNet \gets$ Trained CNN model
$boundingBoxes \gets [\,]$
$detRooms \gets [\,]$
$boundingBoxes \gets$ Evaluate picture using RoomsNet
for each $b \, \mathrm{of} \, boundingBoxes$ do
$t \gets \mathrm{type}(b)$
$detRooms[t] \gets$ Detection probability associated to b
end
return detRooms
end

Algorithm 2. Objects detection algorithm.

Function DetectObjects(picture) is

$ObjectsNet \gets$ Trained CNN model

$boundingBoxes \gets [\,]$

$detObjs \gets [\,]$

$boundingBoxes \gets$ Evaluate picture using ObjectsNet

for each $b \, \mathrm{of} \, boundingBoxes$ do

$t \gets \mathrm{type}(b)$

$detObjs[t] \gets$ Detection probability associated to b

end

return detObjs

end

Although the usage of neural networks was also considered, as it is a system designed to work collaboratively with humans, decision trees were chosen. This technique has the advantage that it models how a person makes decisions much better. Indeed, the probability of a victim is in a given region is determined by successively asking whether a series of objects, ranked from highest to lowest order of importance, are located in that region. It is true that, compared to simple decision trees, random forests are less intuitive but have the ability to perform better with large datasets—with less overfitting—and are able, in this work, to achieve similar results to the ones offered by a neural network.

Model hyperparameters were optimized following a cross-validation search process. Their final values can be found in table 4. In the same way, to increase the model's generalisation ability, variables with significance rates under 1% were removed from it, reducing the number of variables to 13. 70% of mAP precision and 40% of MSE loss were reached. Although those results can be considered low compared to typical Machine Learning accuracy rates, their validity, once the whole system is integrated, will be discussed in the 4 section.

Figure 11 shows the first levels of one of the trees (in a total of 13). As previously mentioned, obtaining the significance level of the different variables is possible. These values explain how much the loss function would increase if this explanatory variable were absent in the model. Figure 12 shows significances for the final trained model.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

The third component required to enable initial exploration was the doors' detector. When ARTU-R arrives in a post-disaster environment, it starts searching for doors, goes to the nearest door and, once through the door, explores the room. Once explored, it should look for new doors in the same room and walk. If no door is found in the room, the robot leaves the room and goes to other doors found in the previous search.

Detecting doors is also done with a CNN, using the dataset [65]. It has 1213 images in which room doors, shelf doors (including drawers), fridge doors and door handles of all types of doors have been labelled. For this purpose, the network was trained for 5 epochs.

Evolution of loss can be seen in figure 13. As can be noticed, once certain peaks are reached, the training is asymptotically stagnant, and the improvements introduced by the new epochs are negligible.

Zoom In Zoom Out Reset image size

Figure 14 illustrates the confusion matrix, demonstrating that doors are correctly identified 50% of the time. In the remaining instances, they are classified as background, implying that they are not assigned a specific category. While this outcome may be viewed as suboptimal, it concurs with the results reported by the dataset's authors [65]. Additionally, it is important to highlight that the probability of any environmental element being misclassified as a door is minimal, given that false positives are as low as 15%.

Zoom In Zoom Out Reset image size

Examples of correct detections of difficult situations—a door with little contrast and a partially-occluded door—are depicted in figure 15.

Zoom In Zoom Out Reset image size

In figures 16(a) and (b), two instances of false positives are observed. These occurrences, as previously mentioned, appear to be unusual situations, and notably, both images feature windows being misclassified as doors, both with low probability detection. The inclusion of windows in the dataset could potentially mitigate this issue.

Zoom In Zoom Out Reset image size

Figure 16(c) presents a scenario in which two room doors are erroneously classified as refrigerator doors. The most significant issue encountered by the system—the non-detection of certain doors-is exemplified in figure 16(d). Instances similar to this one, where the lower portion of the door is obscured from view, are infrequent in scenarios like the one presented in this paper, primarily due to the robot's height.

Once the doors have been identified in the image, the system can locate them. This process is achieved by estimating the relative angular position of the door concerning the robot and comparing the estimate with the obstacles on the map identified with the lidar. Although the CNN only achieves a detection rate of 50%, because each door is photographed, on average, twice, resulting in a high probability of detection. Consequently, ARTU-R stores the positions of all the doors in the environment with errors in the order of centimetres, as it will be commented in the 4.1 section. A pseudocode of the whole process of door detection is shown in algorithm 3.

Algorithm 3. Doors' detection algorithm.
Function SearchForDoors() is
$nPossDoors \gets 15$
$doorTol \gets {60}^{\circ}$
$\boldsymbol{pos} \gets$ robot position
$nAttempts \gets 0$
$doors \gets [\,]$
do
$nAttempts \gets nAttempts + 1$
Rotate doorTol
Take a picture and search for doors
if door detected then
for each door do
Locate the door in the map
$door.dist \gets$ $\left\lVert\boldsymbol{pos} - (door.x, door.y)\right\rVert$
$doors.\mathrm{push}(door.x, door.y, door.dist)$
end
end
while $\mathrm{isEmpty}(doors) \,\, \text{and} \,\, nAttempts \lt nPossDoors$
if $\mathrm{isNotEmpty}(doors)$ then
Go to the nearest door
else
Environment with only one room
end
end

Information from the environment is captured using ARTU-R's embedded lidar sensor. A SLAM module is integrated into ARTU-R's sensor processing architecture and returns a cell map. The map is sent to the master using the /map ROS topic. It consists of a bi-dimensional array in which each element represents the state of one of the cells. A resolution of 0.1 m has been selected according to the size of ARTU-R. The state of the cell is expressed in occupancy probability, from 0 (free) to 1 (occupied).

For the NBV selection process, cells with values lower than 0.5 are considered free. Non-explored cells take the value of −1. The array size is increased during exploration if new regions are detected. The information on obstacles is also shown in RViz interface because it was considered useful for operators, as figure 22 illustrates.

The control of ARTU-R is achieved by sending linear and angular velocity commands to it. Although the robot can be considered holonomic, it has been preferred, for the sake of simplicity, to command only by the two aforementioned components of velocity.

Planning and control are done in three different steps. If only a rotation is required, as when ARTU-R has to take the pictures to evaluate the different NBV candidates, angular velocity commands are sent directly from the master. Their values are adjusted by using a PID control with saturations. The values of the PID parameters can be seen in table 5.

If a translation is required, but to a distance lower than 1.5 m, a similar PID controller is implemented, which parameters can also be found in table 5. What the system does in that situation is to firstly orientate ARTU-R to the goal point and, subsequently, command a straight-line movement till reaches it.

Both linear and angular speed PID controllers are limited to adapt them to the physical characteristics of the robot. For safety reasons, speeds higher than 0.3 m s⁻¹ and 0.6 rad s⁻¹ are not allowed. In the same way, a point or orientation is considered reached when errors lower than 0.3 m and 0.1 rad are measured. Figure 17 presents the block diagram of the controller.

Zoom In Zoom Out Reset image size

In further points, however, planning strategies are needed, although they are more time-consuming because the controller does not guarantee obstacle avoidance. The planner operates from the master, establishing a path between ARTU-R's present position and the desired goal by traversing only cells categorised as free on the map. Once it has been established, velocity commands are sent to the robot. If obstacles are detected, the costmap is updated and the path is locally replanned.

The three developed subsystems have been integrated in order to finally obtain the complete system, whose execution flow is shown in figure 18 and algorithm 4. Table 6 presents the main configuration parameters of the complete algorithm.

Zoom In Zoom Out Reset image size

Algorithm 4. Complete algorithm.
$SearchForDoors()$
for each room do
$picture \gets$ Take a picture
$detRoom \gets DetectRoom(picture)$
while room is not explored do
$NBV \gets SelectNBV(detRoom)$
Move to NBV
end
$SearchForDoors()$
end

As it was explained when arriving at the initial point—the entrance hall of a house, the foyer of an office building, etc–, the system first looks for doors taking an image and applying the appropriate detector to it. If any door is found, ARTU-R goes to the nearest one and explores the room behind it. If none is found, it rotates the angle specified in the parameter doorTol (which is defined in table 6 and repeats the process until a door is found or a complete turnaround has been made. If this happens, it goes to a random point and searches for new doors. If this situation is repeated a number of times nPossDoors, the scenario is considered as one single room (the entrance hall is not only a hall) and it is explored.

When room exploration finishes, ARTU-R starts the same process of door searching inside the room. If more doors are found, the robot will traverse them. If not, it will exit the room and go to the second door of the ones detected in the previous step.

The process of room exploration, on the other hand, starts by defining room type. This step will be done only once until a new room is explored. After that, the system traverses the space moving, at each time, to the NBV point.

When, at step k, robot is at position x _k , it randomly generate a number nPoints of candidates $\boldsymbol{x}_{k+1}^j$ (where $j = 1, \ldots, nPoints$), located at a distance less than dist from x _k . Cells corresponding to candidates must be categorised as 'free' by the SLAM tool which reads the information coming from the laser. All of them receive a fitness value according to equation (1)

$$\begin{align} G(\boldsymbol{x}_{k+1}^{\,j}) & = \omega_{1} \cdot V(\boldsymbol{x}_{k+1}^{\,j}) + \omega_{2} \cdot I(\boldsymbol{x}_{k+1}^{\,j}) \nonumber\\ & \quad + \omega_{3} \cdot d(\boldsymbol{x}_{k}, \boldsymbol{x}_{k+1}^{\,j}) \end{align} \tag{ 1 }$$

The values of $\boldsymbol{\omega} = (\omega_{1}, \omega_{2}, \omega_{3})$ have been determined experimentally and, for this case, are $\boldsymbol{\omega} = (0.1;\, 2;\, -0.7)$.

Equation (1) consists of three terms:

• Explorable volume around the candidate—$V(\boldsymbol{x}_{k+1}^{\,j})$: measured as the percentage of free cells that lie in a square of dimension $(2 \cdot \mathrm{\texttt{candRadius}}) \times (2 \cdot \mathrm{\texttt{candRadius}})$ around $\boldsymbol{x}_{k+1}^{\,j}$. This is a value to maximise because the clearer the surroundings of a point are, the better the chances of detection by the robot's camera. The measurement is made in percentage and not in a total number of points as this will inevitably be lower at the edges of the map.
• Information gain—$I(\boldsymbol{x}_{k+1}^{\,j})$: in order to measure it, the robot is oriented towards $\boldsymbol{x}_{k+1}^{\,j}$, a picture is taken and the different probabilities of having detected an object together with the probabilities associated to the current room are introduced in the trained random forest. The value of $I(\boldsymbol{x}_{k+1}^{\,j})$ is, in consequence, the one returned by the intelligence.
• Distance to point—$d(\boldsymbol{x}_{k}, \boldsymbol{x}_{k+1}^{\,j})$: Euclidean distance measured in norm-2. Associated coefficient ω₃ is negative to prioritise the exploration of closest points.

Equation (1) is similar in its external form to [47]. However, while the latter considers free space and potential information gain and penalises visited points, the model presented here considers, for example, [44] and [46], the distance to the candidate. A pseudocode implementation of the function can be found in algorithm 5.

Algorithm 5. NBV selection algorithm.
Function SelectNBV(detRooms) is
$nPoints \gets 3$
$dist \gets 5$ m
$candRadius \gets 3$ m
$\boldsymbol{\omega} \gets [0.1, 2, -0.7]$
$\boldsymbol{pos} \gets$ robot position
$n \gets 0$
while n $\lt$ nPoints do
do
$\boldsymbol{cand}.x \gets$ random point in interval $[\boldsymbol{pos}.x - dist, \boldsymbol{pos}.x + dist]$
$\boldsymbol{cand}.y \gets$ random point in interval $[\boldsymbol{pos}.y - dist, \boldsymbol{pos}.y + dist]$
while $\mathrm{isNotFree}(\boldsymbol{cand})$
$n \gets n + 1$
end
for each  cand do
$V \gets$ % of unexplored cells in a square of side candRadius cells around cand
Rotate ARTU-R towards cand
$picture \gets$ Take a picture
$detObjs \gets DetectObjects(picture)$
$I \gets$ Random forest evaluation of detRooms and detObjs
$D \gets \left\lVert\boldsymbol{cand} - \boldsymbol{pos}\right\rVert$
if ${human detected in the picture}$ then
Publish an arrow
end
$\boldsymbol{cand}.G \gets \boldsymbol{\omega}_1 \cdot V + \boldsymbol{\omega}_2 \cdot I + \boldsymbol{\omega}_3 \cdot D$
end
$NBV \gets \boldsymbol{cand}$ with higher G
return NBV
end

The victim identification process is executed simultaneously to object identification for calculating information gain. This is because object detector is also able to identify humans. If damaged people are found, an arrow pointing to them is placed on the interface map.

As previously mentioned, the room detector achieved 97% mAP precision over training data. To ensure that such a high accuracy was not due to overfitting, experiments on real images were done using pictures of the different training scenarios.

The qualitative results of the tests—examples of detection can be seen in figure 19—showed, for each class, an accuracy similar to that achieved in training. An empirical probability threshold of 30% was established: when the detection probability is lower than this value, no detection is considered.

Zoom In Zoom Out Reset image size

Human detection is done simultaneously with object detection to obtain a fitness function value. In that task, the system's success rates—working even with mannequins—are around 100 percent, as expected, thanks to the power of CNNs. The system can detect people in all positions: sitting on a chair, squatting, lying down. As figure 20 shows, it is enough to identify a small part of the body—an arm, a leg, the head—for a bounding box to appear, categorising it as a human. The figure also shows that the detector can succeed even with blurred or poorly focused images.

Zoom In Zoom Out Reset image size

Victim detection—and in general, detection systems based on RGB images—tends to fail in poorly lit or excessively dark scenarios. In the illuminance conditions in which the test have taken place—between 400 and 1000 lx—no problems related to victim recognition have been appreciated. To ensure detection in less illuminated spaces, one potential solution could involve equipping the robot with a head-mounted spotlight, capable of generating ample illumination for the captured images.

When victims are identified in the pictures, an arrow pointing to them is placed on the map, which is visible to the system operators, as is shown in figure 21. The position of the origin of the arrow and its orientation are the ones that ARTU-R was at the moment of taking the picture. Although victims are not directly placed on the map—no semantic SLAM nor similar techniques have been used–, the high human recognition rates make it probable to detect the same victim from different points, which means that several arrows will point to her. Interviews with people outside the world of Robotics showed that this solution allows us to identify the places in which victims are located correctly.

Zoom In Zoom Out Reset image size

The evaluation of the door detection process took place in a real environment, in the building used for the validation process. The different tests consisted in leaving the robot to search for doors in this environment, placing them on the map and then checking—the scaled plan of the site, presented in figure 3 was available—whether the placement given by the system coincided with the real one. To do this, different initial points were defined to assess ARTU-R's ability to identify doors in different contexts, such as long distances, the corners of the image, those at dead angles, etc.

Figure 22 shows the accuracy of the subsystem for door detection. In average, a door detection rate of 72% is achieved. The system also has a false positive rate (objects that are not doors are marked as doors) of 6%. and placed with errors of less than 0.92 m.

Zoom In Zoom Out Reset image size

This is clearly one of the major sources of uncertainty in the work presented here. This uncertainty can be attributed to two primary sources of error. Firstly, it pertains to the precision of the Door Detector, as previously discussed. Secondly, it arises from the methodology employed for gate localisation on the map. This approach relies on estimating the gate's angle relative to the robot's position, using a two-step process. Initially, it approximates this angle from a planar photograph and subsequently aligns it with an obstacle within the lidar-generated map. To address the second source of error, refining angle estimations by accounting for image deformation during projection could be a viable solution.

Validation was firstly conducted in the environments presented in section 3.1.1. As specified, two types of orange scenarios were designed, with different difficulty levels. A video of the missions carried out in both of them is available in A.1.

The path followed by ARTU-R during the fist mission is shown in figure 23. As expected, the first thing ARTU-R did was search for doors. Because only one was found during the first series of camera snapshots, it headed straight for that door (P1). After entering in the room (P2), it went behind the desk (P3) because a big unexplored area had been detected and a table had been identified and taken into account, by the random forest, as objects with a high correlation to human presence.

Zoom In Zoom Out Reset image size

The victim placed in that position (V1) was correctly detected and ARTU-R decided to cross to the second room (P4). No exploration around the pallet was done because its transparent nature made it possible to directly check if it harboured any victims behind him.

Exploration of the second room started just after having crossed the door (P5). The first selected NBV was one located near the table and the grey panel (P6), in which another victim (V2) was located and detected by ARTU-R. After that, the robot moved to the background region (P7).

Although no victim was present, it seems natural, even for people, to go there, because it was relatively close to the previous position and there was a lack of visibility. The last explored point was the bottom left corner of the room (P8), where the last remaining human (V3) was found.

In terms of time, the whole test took, for exploring an area of 49 m² (7.6 m × 6.5 m), 4 min and 15 s (255 s), which makes an exploration speed of 0.192 m² s⁻¹. Statistics related to this test can be found in figure 24.

Zoom In Zoom Out Reset image size

The necessity of finding an equilibrium between exploration and exploitation to solve the problem is clearly displayed in the graph. It is very easy to see that when ARTU-R enters a new room (when the door is crossed), the room is rapidly mapped, achieving exploration speeds greater than 0.5 m² s⁻¹. Once this process concludes, the system starts moving inside the room looking for victims. In both rooms, victims have been found relatively early, which shows the importance of using the information provided by the camera in the process, and the good performance of the intelligence system.

Figure 25 presents the map of the environment reconstructed by ARTU-R at different steps of the mission. As previously commented, it is important to note that all the hidden victims were correctly located during the test.

Zoom In Zoom Out Reset image size

The test was repeated 10 times. Durations of the mission ranged from 212 to 308 s. The route taken by the robot, while not precisely identical, closely resembles the route shown in figure 23, always exploring from very close points. It is worth noting, however, that in 3 of the 10 repetitions, the second room's exploration sequence was P8-P9-P6-P7. However, this can be considered logical as, although there were no objects correlated with the presence of humans near P8, there was, upon entering the room, a substantial amount of unexplored space around it.

The test environment layout for the second battery of tests is depicted in figure 26. It contains two ramps at the entrance. One is an ascending ramp with an initial step height of 14 cm, a length of 87.5 cm and a slope of 11.8^∘. The other is a descending ramp with a slope of 12.4^∘.

Zoom In Zoom Out Reset image size

Additionally, a beam, which is an insurmountable obstacle in the form of a step for medium-sized wheeled robots, has been placed between points P6 and P7. The passages between P3 and P4 and P4 and P5 are narrow and require a robot that is not too wide and has relatively good manoeuvrability.

Path followed by ARTU-R explores sequentially the scene. Once the ramp had been overcome (P2) it looked for objects of interest (P3) and found a table. The area behind it was explored (P4) and a victim (V1) was found. The robot continues its moving up to the next table (P5-P6), in where another victim was found (V2). Finally, the beam was traversed (P7) and the last victim (V3), found. No possibility of finding that last victim would have been possible without access to the latter region.

The test was repeated 5 times. The victim search sequence was the same in all of them. In average, they took a time, measured from P0 to the victim's location V3, of 5 min and 37 s (337 s).

A second validation process has been done in the real environment of section 3.1.1. The test rescue mission started at the P0 point of figures 27(a) and (b), which corresponds to the entrance of the building. Door detection process yields the detection of three doors, the open door of figure 27(a), which gives entrance to a computer room, the corridor of the same figure, and the door of figure 27(b), that gives access to a small and plenty of obstacles lab.

Zoom In Zoom Out Reset image size

As expected, the system decided to start exploring the region behind the nearest door, corresponding to the one delimiting the computer room. The path ARTU-R follows to reach the door is represented in red (P1-P3). After crossing it, ARTU-R rapidly explored the room: a few steps were enough to cover the entire region and detect the human placed there.

After that, an exploration of the corridor, which path is represented in blue in figure 27(a)(P3-P7), was done. ARTU-R's behaviour during that part of the mission was similar to that of a human would have: it walked down forward, discarded to enter in the open rooms because visibility at the entrance was enough, and returned once the end was reached.

The third part of the mission is reproduced in figure 27(b). The system looked for victims in the lab located behind the door (P9). Although its many objects can difficult the exploration, as figure 28 shows, ARTU-R was able to replan its paths to avoid collisions, keeping the objective to reach the selected NBV. When the room was explored, no other doors were detected, and the mission was considered completed (P10-P11).

Zoom In Zoom Out Reset image size

The complete test took a time of 18 min and 30 s (1110 s) to explore an area of 225 m², which gives an exploration speed of 0.203 m² s⁻¹.

As previously stated, the testing scenario was relatively uncomplicated. Our goal here was not to evaluate the system's effectiveness in the aftermath of significant disasters like earthquakes, but rather in situations such as gas leaks, which do not lead to structural damage. Consequently, this particular test was conducted only once.