TIP: A trust inference and propagation model in multi-human multi-robot teams

Abstract

Trust is a crucial factor for effective human–robot teaming. Existing literature on trust modeling predominantly focuses on dyadic human-autonomy teams where one human agent interacts with one robot. There is little, if not no, research on trust modeling in teams consisting of multiple human and robotic agents. To fill this important research gap, we present the Trust Inference and Propagation (TIP) model to model and estimate human trust in multi-human multi-robot teams. In a multi-human multi-robot team, we postulate that there exist two types of experiences that a human agent has with a robot: direct and indirect experiences. The TIP model presents a novel mathematical framework that explicitly accounts for both types of experiences. To evaluate the model, we conducted a human-subject experiment with 15 pairs of participants ($N=30$). Each pair performed a search and detection task with two drones. Results show that our TIP model successfully captured the underlying trust dynamics and significantly outperformed a baseline model. To the best of our knowledge, the TIP model is the first mathematical framework for computational trust modeling in multi-human multi-robot teams.

1 Introduction

The collaboration between humans and robots is rapidly advancing, with robots being deployed in diverse fields such as urban search and rescue (USAR) (Murphy, 2004), manufacturing (Unhelkar et al., 2014), and healthcare (Rantanen et al., 2017; Gombolay et al., 2018), among others. These robotic agents are increasingly being used to perform complex tasks, such as advising on resource allocation strategies in labor and delivery units (Gombolay et al., 2018). The success of such teamwork, however, critically depends on the trust that humans, such as physicians, have in these robotic systems. Without this trust, the integration and effectiveness of robots in critical settings remain limited. Trust-aware human-robot interaction is an area under active research, where a person’s trust in a robotic agent is explicitly incorporated in robot planning and decision-making (Bhat et al., 2022; Chen et al., 2018, 2020; Xu and Dudek, 2016; Guo et al., 2021).

While significant efforts have been made in understanding trust within dyadic partnerships of one human and one robot (National Academies of Sciences, Engineering, and Medicine, 2022), the dynamics of trust in multi-human multi-robot settings is largely unexplored. Especially, as advancements in artificial intelligence (AI) and robotics pave the way for workplaces where multiple humans and robots collaborate extensively, the necessity for understanding and modeling trust in such multi-agent environments grows even more critical. This is particularly relevant in sectors demanding advanced coordination and adaptability, like military operations (Ramchurn et al., 2015; Freedy et al., 2008), factory automation (Liu et al., 2021), and automated agriculture systems (Lippi et al., 2023; Ji et al., 2022). The complexity introduced by the dynamic compositions of human-agent teams requires a refined understanding of trust dynamics in multi-agent human–robot interactions to achieve seamless collaboration.

Consider a scenario where two human agents (Fig. 1a), x and y, and two robots, A and B, are to perform a task. The four agents are allowed to form sub-teams to enhance task performance (e.g., maximizing throughput and minimizing task completion time). For instance, they could initially form two dyadic human–robot teams to complete the first two tasks, and then reconfigure to complete the third task, and so on. This scenario illustrates a new organizational model known as “team of teams (Meehan and Jonker, 2018; McChrystal et al., 2015)” in which the team composition is fluid and team members come and go as the nature of the problem changes.

Fig. 1

Trust propagation in multi-human multi-robot teams (Color figure online)

In this scenario, we postulate that there exist two types of experiences that a human agent has with a robot: direct and indirect experiences. Direct experience, by its name, means that a human agent’s interaction with a robot is by him-/her-self; indirect experience means that a human agent’s interaction with a robot is mediated by another party. Considering the third task (see Fig. 1), human x works directly with robot B (i.e., direct experience). Even though there is no direct interaction between x and A in part 3, we postulate that x could still update his or her trust in A by learning her human teammate y’s experience with A, i.e., y’s direct experience with A becomes x’s indirect experience with A, based on which x can update her trust in A, $t^{x, A}$. Essentially, y’s trust in A propogates to x.

Under the direct and indirect experience framework, prior work on trust modeling in dyadic human–robot teams can be regarded as examining how direct experience influences a person’s trust in a robot. In multi-human multi-robot teams, we postulate that both direct and indirect experiences drive a human agent’s trust in a robot.

To model trust dynamics in such multi-agent setting, we develop the Trust Inference and Propagation (TIP) model for multi-human multi-robot teams. The proposed model explicitly accounts for both the direct and indirect experiences a human agent may have with a robot. We examine trust dynamics under the TIP framework and prove theoretically that trust converges after repeated (direct and indirect) interactions. To evaluate the proposed TIP model, we conducted a human-subject experiment with 15 pairs of participants ($N=30$). Each pair worked with two drones to perform a threat detection task for 15 sessions. We compared the TIP model (i.e., accounts for both the direct and indirect experiences) and a direct-experience-only model (i.e., only accounts for the direct experience a human agent has with a robot). Results show that the TIP model successfully captures people’s trust dynamics with a significantly smaller root-mean-square error (RMSE) compared to the direct-experience-only model.

The key contribution of this work is three-fold:

• To the best of our knowledge, the proposed TIP model is the first mathematical framework for computational trust modeling in multi-human multi-robot teams. The TIP model accounts for both the direct and indirect experiences (through trust propagation) a human agent has with a robot in multi-human multi-robot teams. As a result, the TIP model is well-suited for trust estimation in networks involving multiple humans and robots.

• We prove theoretically that trust converges to the unique equilibrium in probability after repeated direct and indirect interactions under our TIP framework. Such an equilibrium can also be efficiently computed.

• We conduct a human-subject experiment to assess the TIP model. Results reveal the superior performance of the TIP model in capturing trust dynamics in a multi-human multi-robot team.

This paper is organized as follows. Section 2 presents related work, including trust modeling in dyadic human–robot teams and reputation/trust management in e-commerce. In Sect. 3, we describe the mathematical formulation of the TIP model and examine its behavior under different types of interactions. Section 4 presents the human-subject study. In Sect. 5, we present and discuss the results. Section 6 concludes the paper.

2 Related work

In this section, we review two bodies of research motivating the present study: the extensive literature on trust in dyadic human–robot teams and the literature on reputation/trust management. The latter is a research topic in computer science that shares commonalities with the underlying research question of trust modeling in multi-human multi-robot teams.

We note that “trust" has been extensively studied across various fields and has been given different definitions (Hawley, 2012; Tavani, 2015; Cho et al., 2015; Coeckelbergh, 2012; Kok and Soh, 2020). In this work, we focus on trust in automation and use the definition provided by Lee and See (2004). This definition emphasizes the uncertain nature of HRI and the vulnerability inherent in a human trusting a robot during collaboration, which aligns with our research objectives.

2.1 Trust modeling in Dyadic human–robot interaction

Trust in autonomous/robotic agents attracts research attention from multiple disciplines. One line of research is to identify factors influencing a human’s trust in autonomy/robots and quantify their effects. These factors can be categorized into human-related factors such as personality (Bhat et al., 2022), robot-related factors such as reliability (Lyons et al., 2021; Gombolay et al., 2018) and transparency (Wang et al., 2016; Luo et al., 2022), and task-related factors such as task emergency (Robinette et al., 2016). For a review of the factors, see Hancock et al. (2021). More recently, another line of research has emerged that focuses on understanding the dynamics of trust formation and evolution when a person interacts with autonomy repeatedly (Yang et al., 2021; Visser et al., 2020; Guo and Yang, 2020). Empirical studies have investigated how trust strengthens or decays due to moment-to-moment interactions with autonomy (Lee and Moray, 1992; Manzey et al., 2012; Yang et al., 2021, 2017). Based on the empirical research, three major properties of trust dynamics have been identified and summarized, namely continuity, negativity bias, and stabilization (Guo and Yang, 2020; Yang et al., 2023).

Acknowledging that trust is a dynamic variable, several computational trust models in dyadic human–robot teams have been developed (Chen et al., 2018; Xu and Dudek, 2015; Guo and Yang, 2020; Wang et al., 2016; Bhat et al., 2024). Notably, Xu and Dudek (2015) proposed the online probabilistic trust inference model (OPTIMo) utilizing Bayesian networks to estimate human trust based on the autonomous agent’s performance and human behavioral signals. Guo and Yang (2020) modeled trust as a Beta random variable parameterized by positive and negative interaction experience a human agent has with a robotic agent. Soh et al. (2020) proposed a Bayesian model which combines Gaussian processes and recurrent neural networks to predict trust over different tasks. For a detailed review, refer to Kok and Soh (2020).

2.2 Reputation/credential management

Despite limited research on trust modeling in multi-human multi-robot teams, insights can be drawn from studies on reputation management. In consumer-to-consumer electronic marketplaces like eBay, reputation systems play a crucial role in generating trust among buyers to facilitate transactions with unknown sellers (Dellarocas, 2003). These systems can be categorized as centralized, where reputation values are stored centrally, representing the overall trustworthiness of sellers, or decentralized, where buyers maintain their evaluation scores privately (Hendrikx et al., 2015). In decentralized systems, a propagation mechanism allows buyers to obtain reputation values, even in the absence of prior transactions. Various propagation mechanisms have been developed, such as subjective logic integrated into the Beta reputation management system (Jøsang, 1997; Josang and Ismail, 2002) or the concept of "witness reputation" in the FIRE reputation management model (Huynh et al., 2004), facilitating the transfer of reputation scores among agents in a network. These propagation mechanisms provide valuable insights into modeling trust updates through indirect experience in HRI. Yet, their direct application to HRI settings is impeded due to the different characteristics between human-to-robot trust and human-to-human trust (Kessler et al., 2017), as reviewed in Sect. 2.1.

3 Mathematical model

We present the TIP model in this section. Our key motivation is to develop a fully computational trust inference and propagation model that works in general multi-human multi-robot settings. First, we define two key concepts, “trust” and “interaction”, that are central to this study, as they have different interpretations in different contexts. Second, we discuss the assumptions and introduce the mathematical formulation. Third, we examine the behavior of the model under repeated human–robot interactions. Finally, we present the parameter inference method and trust estimation using the TIP model.

3.1 Definition of trust and interaction

We use the definition of trust given by Lee and See (2004): trust is “the attitude that an agent will help achieve an individual’s goals in situations characterized by uncertainty and vulnerability”. This conceptualization underscores three critical dimensions of trust in HRI. First, it emphasizes the goal-oriented nature of trust, where trust is directed towards the achievement of specific objectives. Second, it acknowledges the inherent uncertainty that pervades interactions between humans and robots, a fundamental aspect of our study. Lastly, it encapsulates the notion of reliance, highlighting how humans depend on robots within a collaborative team context.

In this study, we focus on episodic human–robot interaction. The duration of such an episode depends on the specific task or scenario and can vary based on the resolution of the analysis. For example, a single episode may encompass a complete task cycle, such as a robot assisting in a search-and-rescue mission, where the episode duration is determined by the mission length. Alternatively, in a more granular analysis, an episode could be as brief as a single interaction within a larger task, like a robot handing over a tool to a human in a manufacturing setting, where each tool exchange constitutes an individual episode. This flexibility in defining the episode length allows for a detailed examination of trust dynamics at varying levels of interaction complexity and duration.

3.2 Assumptions

We make three major assumptions in the context of HRI. First, we assume that each human agent communicates trust as a single-dimensional value. In some prior work, trust is represented as a tuple. For example, Josang and Ismail (2002) modeled trust as a triplet, i.e., belief, disbelief, and uncertainty. Although a multi-dimensional representation conveys more information, our study as well as some prior studies show that a one-dimensional representation of trust suffices in capturing trust evolution  (Chen et al., 2018; Xu and Dudek, 2015; Wang et al., 2016; Guo and Yang, 2020; Guo et al., 2021). Moreover, querying a single-dimension trust value increases operational feasibility because keeping track of multiple numbers adds unnecessary cognitive load and may not be pragmatic for non-experts. Therefore, we assume a simple one-dimension form of trust in this study.

Second, we assume that the human agents are cooperative, i.e., they are honest and willing to share their trust in a robot truthfully with their human teammates. This assumption directly affects our model and experiment design. According to Mayer et al. (1995), the bases of trust include the trustee’s ability, integrity, and benevolence. With the cooperation assumption, a trustee’s ability is the major factor affecting the trustor’s trust. As a consequence, the trustor’s trust evolves only when the trustor’s perceived ability of the trustee changes. Therefore, in the experiment design, we ask the participants to report trust in their teammate “based on the teammate’s ability to rate trust in the drone” (Sect. 4.2). Moreover, as the trustee is honest, when they inform the trustor of their trust in a robot, the trustor may discount this information but will not attempt to reverse it. This motivates us to use between-human trust as a discount factor for trust propagation.

Third, we take an ability/performance-centric view of trust and assume that a human agent’s trust in a robot is primarily driven by the ability or performance of the robot. Based on the trust model of Mayer et al. (1995) and the premise that the robots are not deceptive, human trust in robots is primarily determined by robots’ performance. In addition, this ability/performance-centric view has been widely used in prior research for modeling trust in task-oriented HRI contexts (Pippin and Christensen, 2014; Xu and Dudek, 2015; Chen et al., 2018; Guo et al., 2023. i.e., a robot is to perform a specific task)

3.3 Proposed model

We summarize the notation used throughout the paper in Table 1. The notation system follows the convention in Fig. 1b. A variable with superscript a, b indicates that the variable represents a relation from trustor a to trustee b, and subscript k indices the episodic interactions.

Table 1 Notation table

Trust as a Beta random variable. We take a probabilistic view to model trust as in Guo and Yang (2020). At time k, the trust $t_{k}^{a,b}$ that a human agent a feels toward another agent b follows a Beta distribution, i.e.,

$$\begin{aligned} t_{k}^{a,b} \sim {\text {Beta}}\left( \alpha _{k}^{a,b},\beta _{k}^{a,b}\right) \text {,} \end{aligned}$$

(1)

where $\alpha _{k}^{a,b}$ and $\beta _{k}^{a,b}$ are the positive and negative experiences a had about b up to time k, respectively, $k=0,1,2,\dots$. When $k=0$, $\alpha _{0}^{a,b}$ and $\beta _{0}^{a,b}$ represent the prior experiences that a has before any interaction with b. The expected trust is given by

$$\begin{aligned} \mu _{k}^{a,b} =\alpha _{k}^{a,b} /\left( \alpha _{k}^{a,b} +\beta _{k}^{a,b}\right) \text {.} \end{aligned}$$

(2)

Here we note that $t_{k}^{a,b}$ is the queried trust given by the agent a, which has some randomness due to subjectivity, while $\mu _{k}^{a,b}$ is the expected trust determined by the experiences.

Trust update through direct experience. We update the experiences through direct interaction at time k by setting

$$\begin{aligned} \begin{aligned} \alpha _{k}^{a,b}&=\alpha _{k-1}^{a,b} +s^{a,b} \cdot p_{k}^{b}\\ \beta _{k}^{a,b}&=\beta _{k-1}^{a,b} +f^{a,b} \cdot \overline{p}_{k}^{b} \end{aligned}\text {.} \end{aligned}$$

(3)

Here $p_{k}^{b}$ and $\overline{p}_{k}^{b}$ are the measurements of b’s success and failure during time k, respectively. The incremental from $\alpha _{k-1}^{a,b}$ to $\alpha _{k}^{a,b}$ is the positive experience gain that agent a learns based on b’s performance in the kth interaction; similarly, the increment from $\beta _{k-1}^{a,b}$ to $\beta _{k}^{a,b}$ represents the negative experience gain. $s^{a,b}$ and $f^{a,b}$ are a’s unit experience gains with respect to success or failure of b. We require $s^{a,b}$ and $f^{a,b}$ to be positive to ensure that cumulative experiences are non-decreasing. The updated trust $t_{k}^{a,b}$ follows the distribution ${\text {Beta}} (\alpha _{k}^{a,b},\beta _{k}^{a,b})$.

Trust update through indirect experience propagation. Let x and y denote two human agents and let A denote a robot agent, as illustrated in Fig. 1b. At time k, y communicates his or her trust $t_{k}^{y,A}$ in A with x, and then x updates his or her experiences through indirect interaction by

$$\begin{aligned} \begin{aligned} \alpha _{k}^{x,A}&=\alpha _{k-1}^{x,A} +\hat{s}^{x,A} \cdot t_{k}^{x,y} \cdot \left[ t_{k}^{y,A} -t_{k-1}^{x,A}\right] ^{+}\\ \beta _{k}^{x,A}&=\beta _{k-1}^{x,A} +\hat{f}^{x,A} \cdot t_{k}^{x,y} \cdot \left[ t_{k-1}^{x,A} -t_{k}^{y,A}\right] ^{+} \end{aligned}\text {,} \end{aligned}$$

(4)

where the superscript ‘$+$’ means taking the positive part of the corresponding number, i.e., $t^+=\max \{0,t\}$ for a real number t, and $t_{k}^{x,A}\sim {\text {Beta}} ( \alpha _{k}^{x,A},\beta _{k}^{x,A} )$.

The intuition behind this model is that x needs to reason upon $t_{k}^{y,A}$, i.e., y’s trust toward A. First, x compares y’s trust $t_{k}^{y,A}$ with his or her previous trust $t_{k-1}^{x,A}$. Let $\Delta t:=t_{k}^{y,A} -t_{k-1}^{x,A}$ be the difference. If $\Delta t \ge 0$, x gains positive indirect experience about A, which amounts to the product of the trust difference $\Delta t$, a coefficient $\hat{s}^{x,A}$, and a discounting factor $t_{k}^{x,y}$, i.e., x’s trust in y; if $\Delta t<0$, then x gains negative indirect experience about A, which is defined similarly. As we noted in Sect. 3.2, because we assume the humans are cooperative, the between-human trust is treated as a discount factor for trust propagation.

3.4 Asymptotic behavior under repeated interactions

The proposed model allows us to investigate and approximate long-horizon trust dynamics that can be difficult to learn through human-subject study. We examine the behavior of the proposed model under both direct and indirect trust updates. Consider a scenario where human agents x and y take turns working with robot A repetitively. Suppose each x’s turn contains m interactions while each y’s turn contains n interactions; and, after each interaction, the agent who works directly with A informs the other agent of his or her trust in A. Figure 2 illustrates the interaction process. In addition, we assume that robot A has constant reliability r, i.e., A’s performance measure are $p_{k}^A =r$ and $\overline{p}^{A}_k = \bar{r}=1-r$, for $k=1,2,\dots ,K$, and x has constant trust $t^{x,y}$ in y. To avoid triviality, we exclude the case when $m=n=0$ (where no interactions occur). Without loss of generality, we assume $m>0$ and $n\geqslant 0$. (The case $m\geqslant 0$ and $n>0$ is symmetric.)

Fig. 2

x and y take turns to interact with A

We have the following main result on the asymptotic behavior of $t_{k}^{x,A}$ and $t_{k}^{y,A}$.

Theorem 1

When $m >0$ and $n\geqslant 0$, $t_{k}^{x,A}$ and $t_{k}^{y,A}$ converge in probability (i.p.) respectively, i.e., there exists $t^{x}$ and $t^{y}$ such that, for any $\epsilon >0$,

$$\begin{aligned} \begin{aligned} & \lim _{k\rightarrow \infty } \Pr \left( \left| t_{k}^{x,A} -t^{x}\right|>\epsilon \right) =0\ \\ \text {and} & \lim _{k\rightarrow \infty } \Pr \left( \left| t_{k}^{y,A} -t^{y}\right| >\epsilon \right) =0. \end{aligned} \end{aligned}$$

Theorem 1 exhibits that, under alternating interactions with the robot, both agents’ trust will stabilize and converge after sufficiently many interactions. The next result gives an exact method to compute the limiting equilibrium.

Theorem 2

The equilibrium $t^{x}$ and $t^{y}$ in Theorem 1 satisfy

$$\begin{aligned} \begin{aligned} S^{x}\frac{1-t^{x}}{t^{x}}&=\hat{F}^{x}\left( t^{x} -t^{y}\right) +F^{x}&\text {and}\\ F^{y}\frac{t^{y}}{1-t^{y}}&=\hat{S}^{y}\left( t^{x} -t^{y}\right) +S^{y},&\end{aligned} \end{aligned}$$

(5)

if $S^{x} F^{y} \geqslant F^{x} S^{y}$; otherwise, they satisfy

$$\begin{aligned} \begin{aligned} F^{x}\frac{t^{x}}{1-t^{x}}&=\hat{S}^{x}\left( t^{y} -t^{x}\right) +S^{x}&\text {and}\\ S^{y}\frac{1-t^{y}}{t^{y}}&=\hat{F}^{y}\left( t^{y} -t^{x}\right) +F^{y},&\end{aligned} \end{aligned}$$

(6)

where $\hat{S}^{x} =nt^{x,y}\hat{s}^{x,A}$, $\hat{F}^{x} =nt^{x,y}\hat{f}^{x,A}$, $S^{x} =ms^{x,A} r$, $F^{x} =mf^{x,A}\overline{r}$, $\hat{S}^{y} =mt^{y,x}\hat{s}^{y,A}$, $\hat{F}^{y} =mt^{y,x}\hat{f}^{y,A}$, $S^{y} =ns^{y,A} r$, and $F^{y} =nf^{y,A}\overline{r}$.

The capitalized variables in Theorem 2 are related to the average experience gains in the long run, e.g., ${S}^{x}$ is x’s direct positive experience gain after each m direct update. The condition $S^{x} F^{y} \geqslant F^{x} S^{y}$ can be interpreted as follows: compared with y, x tends to have a higher trust gain in A after each turn via direct experience. Note that $t^x$ and $t^y$ can be computed exactly by solving a cubic equation or readily approximated by Newton’s method. Details are given in the appendix.

A special case is when $n=0$, i.e., agent x only updates trust in A via direct experience, and agent y only updates trust via indirect experience. Theorem 2 leads to the following corollary with a closed-form equilibrium:

Corollary 1

When $m >0$ and $n=0$, x’s trust $t_{k}^{x,A}$ in A converges to $t^{x} =\frac{s^{x,A} r}{f^{x,A}\overline{r} +s^{x,A} r}$ in probability, i.e., for any $\epsilon >0$,

$$\begin{aligned} \lim _{k\rightarrow \infty } \Pr \left( \left| t_{k}^{x,A} -\frac{s^{x,A} r}{f^{x,A}\overline{r} +s^{x,A} r}\right| >\epsilon \right) =0. \end{aligned}$$

The difference between $t_{k}^{x,A}$ and $t_{k}^{y,A}$ converge to 0 in probability, i.e., for any $\epsilon >0$,

$$\begin{aligned} \lim _{k\rightarrow \infty } \Pr \left( \left| t_{k}^{x,A} -t_{k}^{y,A}\right| >\epsilon \right) =0. \end{aligned}$$

Equivalently, we have $t^{x} =t^{y}$ in Theorem 2.

Corollay 1 implies that under direct-only trust updates, x’s trust will stabilize around the following closed-form

$$\begin{aligned} \frac{s^{x,A} r}{f^{x,A}\overline{r} +s^{x,A} r}, \end{aligned}$$

which is determined by x’s unit experience gains $s^{x,A}$ and $f^{x,A}$ via direct trust, and the robot’s reliability r; moreover, under indirect-only updates, y’s trust will converge to x’s trust.

3.5 Parameter Inference

The proposed model characterizes a human agent’s trust in a robot by six parameters. For instance, the parameter of x on robot A, which is defined as

$$\begin{aligned} \varvec{\theta } ^{x,A} =\left( \alpha _{0}^{x,A},\beta _{0}^{x,A},s^{x,A},f^{x,A},\hat{s}^{x,A},\hat{f}^{x,A}\right) , \end{aligned}$$

(7)

includes x’s prior experiences $\alpha _{0}^{x,A}$ and $\beta _{0}^{x,A}$, the unit direct experience gains $s_{A}^{x}$ and $f_{A}^{x}$, and the unit indirect experience gains $\hat{s}_{A}^{x}$ and $\hat{f}_{A}^{x}$. Denote the indices of x’s direct and indirect interactions with A up to time k as $D_{k}$ and $\overline{D}_{k}$. We can compute $\alpha _{k}^{x,A}$ and $\beta _{k}^{x,A}$, according to Eqs. (3) and (4), as

$$\begin{aligned} \begin{aligned} \alpha _{k}^{x,A} =&\alpha _{0}^{x,A} +s^{x,A}\sum _{j\in D_{k}} p_{j}^{A} +\hat{s}\sum _{j\in \overline{D}_{k}} t_{j}^{x,y}\left[ t_{j}^{y,A} -t_{j-1}^{x,A}\right] ^{+}\\ \beta _{k}^{x,A} =&\beta _{0}^{x,A} +f^{x,A}\sum _{j\in D_{k}}\overline{p}_{j}^{A} +\hat{f}\sum _{j\in \overline{D}_{k}} t_{j}^{x,y}\left[ t_{j-1}^{x,A} -t_{j}^{y,A}\right] ^{+} \end{aligned}.\nonumber \\ \end{aligned}$$

(8)

We compute the optimal parameter $\varvec{\theta } ^{x,A}_{*}$ by maximum likelihood estimation (MLE), i.e.,

$$\begin{aligned} \begin{aligned} \varvec{\theta }^{x,A}_{*} =&\arg \max \log \Pr \left( \text {data}\Bigg | \varvec{\theta } ^{x,A}\right) \\ =&\arg \max \sum _{k=0}^{K}\log {\text {Beta}}\left( t_{k}^{x,A}\Bigg | \alpha _{k}^{x,A},\beta _{k}^{x,A}\right) \text {.} \end{aligned} \end{aligned}$$

Specifically, the problem of estimating x’s parameter $\varvec{\theta } _{*}^{x,A}$ on robot A is formulated as follows: given x’s full trust history in A, $\{t_{k}^{x,A}\}_{k=0}^K$, A’s performance history during x’s direct trust update in A, $\{( p_{k}^{A},\overline{p}_{k}^{A})\}_{k\in D_{K}}$, x’s trust in y during x’s indirect trust update in A, $\{t_{k}^{x,y}\}_{k\in \overline{D}_{K}}$, and y’s trust in A during x’s indirect trust update in A, $\{t_{k}^{y,A}\}_{k\in \overline{D}_{K}}$, we compute the parameter $\varvec{\theta } _{*}^{x,A}$ that maximizes the log likelihood function

$$\begin{aligned} H( \varvec{\theta } ^{x,A}):=\sum _{k=0}^{K}\log {\text {Beta}}\left( t_{k}^{x,A}\Bigg | \alpha _{k}^{x,A},\beta _{k}^{x,A}\right) , \end{aligned}$$

(9)

where $\alpha _{k}^{x,A}$ and $\beta _{k}^{x,A}$ are defined in Eq. (8).

We note that $\log {\text {Beta}}( t_{k}^{x,A} | \alpha _{k}^{x,A},\beta _{k}^{x,A})$ is concave in $\varvec{\theta } ^{x,A}$ because it is concave in $( \alpha _{k}^{x,A},\beta _{k}^{x,A})$ and $\alpha _{k}^{x,A}$ and $\beta _{k}^{x,A}$ are non-decreasing linear functions of $\varvec{\theta } ^{x,A}$. Consequently, $H( \varvec{\theta } ^{x,A})$ is concave in $\varvec{\theta } ^{x,A}$ since it is a summation of several concave functions. Therefore, we can run the gradient descent method to compute the optimal parameters.

Now we explicitly give the formulas for the gradient descent method. By expressing the probability density function of Beta random variables in terms of Gamma functions, we can rewrite Eq. (9) as

$$\begin{aligned} \begin{aligned}&H( \varvec{\theta }^{x,A} )\\ =&\sum _{k=0}^{K}\left[ \log \Gamma \bigl ( \alpha _{k}^{x,A} +\beta _{k}^{x,A}\bigr ) -\log \Gamma \bigl ( \alpha _{k}^{x,A}\bigr ) -\log \Gamma \bigl ( \beta _{k}^{x,A}\bigr )\right. \\&\left. +\bigl ( \alpha _{k}^{x,A} -1\bigr )\log t_{k}^{x,A} +\bigl ( \beta _{k}^{x,A} -1\bigr )\log \bigl ( 1-t_{k}^{x,A}\bigr )\right] , \end{aligned} \end{aligned}$$

where $\Gamma (\cdot )$ stands for the Gamma function. Define the following variables:

$$\begin{aligned} \begin{aligned} P_{k}&:=\sum _{j\in D_{k}} p_{j}^{A},&Q_{k}&:=\sum _{j\in \overline{D}_{k}} t_{j}^{x,y}\left[ t_{j}^{y,A} -t_{j-1}^{x,A}\right] ^{+},\\ \overline{P}_{k}&:=\sum _{j\in D_{k}}\overline{p}_{j}^{A},&\overline{Q}_{k}&:=\sum _{j\in \overline{D}_{k}} t_{j}^{x,y}\left[ t_{j-1}^{x,A} -t_{j}^{y,A}\right] ^{+}. \end{aligned} \end{aligned}$$

Then (8) becomes

$$\begin{aligned} \begin{aligned} \alpha _{k}^{x,A} =&\alpha _{0}^{x,A} +s^{x,A} P_{k} +\hat{s}^{x,A} Q_{k}\\ \beta _{k}^{x,A} =&\beta _{0}^{x,A} +f^{x,A}\overline{P}_{k} +\hat{f}^{x,A}\overline{Q}_{k} \end{aligned}. \end{aligned}$$

Calculation shows the gradient can be written as

$$\begin{aligned} \nabla H( \varvec{\theta } ^{x,A}) =\sum _{k=0}^{K} {\textbf {C}}_{k} {\textbf {v}}_{k}, \end{aligned}$$

(10)

where

$$\begin{aligned} {\textbf {C}}_{k} =\begin{bmatrix} 1 & -1 & 0 & 1 & 0\\ 1 & 0 & -1 & 0 & 1\\ P_{k} & -P_{k} & 0 & P_{k} & 0\\ \overline{P}_{k} & 0 & -\overline{P}_{k} & 0 & \overline{P}_{k}\\ Q_{k} & -Q_{k} & 0 & Q_{k} & 0\\ \overline{Q}_{k} & 0 & -\overline{Q}_{k} & 0 & \overline{Q}_{k} \end{bmatrix} \end{aligned}$$

(11)

and

$$\begin{aligned} {\textbf {v}}_{k} =\begin{bmatrix} \psi \left( \alpha _{k}^{x,A} +\beta _{k}^{x,A}\right) \\ \psi \left( \alpha _{k}^{x,A}\right) \\ \psi \left( \beta _{k}^{x,A}\right) \\ \log t_{k}^{x,A}\\ \log \left( 1-t_{k}^{x,A}\right) \end{bmatrix}. \end{aligned}$$

(12)

Here $\psi$ is the digamma function. Note that ${\textbf {C}}_{k}$ is constant throughout the gradient descent while ${\textbf {v}}_{k}$ needs to be computed in every iteration.

3.6 Trust estimation

In real HRI scenarios, querying human trust after every interaction is impractical as it introduces extra workload and reduces collaboration efficiency. Instead, we consider the case when human trust is only queried after some, but not all, of the interactions. In particular, we are interested in referring the model parameter $\varvec{\theta } ^{x,A}$ defined in Eq. (7) with missing trust values and estimating these missing values with the TIP model.

Specifically, the input of the trust estimation problem is the same as the parameter inference problem in Sect. 3.5, except that $t_{u}^{x,A}$, $t_{u}^{x,y}$, and $t_{u}^{y,A}$ are missing for $u\in U$, where U is the collection of interactions without trust ratings. We assume $0\notin U$, that is, the initial trust ratings, $t_{0}^{x,y}$, $t_{0}^{y,A}$, and $t_{0}^{x,A}$, are known. The optimal parameter is defined as the maximizer of the log-likelihood given the available data:

$$\begin{aligned} H_{U}( \varvec{\theta } ^{x,A}):=\sum _{k\in \{0,\dotsc ,K\} \backslash U}\log {\text {Beta}}\left( t_{k}^{x,A}\Bigg | \alpha _{k}^{x,A},\beta _{k}^{x,A}\right) . \end{aligned}$$

Equation (8) implies that computing the experiences $\alpha _{k}^{x,A}$ and $\beta _{k}^{x,A}$ relies on the trust ratings $t_{j}^{x,y}$, $t_{j}^{y,A}$, and $t_{j}^{x,A}$. We approximate them by the following recursive relations:

$$\begin{aligned} \hat{t}_{j}^{x,y} =t_{j}^{x,y},\ \hat{t}_{j}^{y,A} =t_{j}^{y,A},\ \text {and} \ \hat{t}_{j}^{x,A} =t_{j}^{x,A}, \end{aligned}$$

for $j\notin U$;

$$\begin{aligned} \hat{t}_{j}^{x,y} =t_{j'}^{x,y},\ \hat{t}_{j}^{y,A} =t_{j'}^{y,A},\ \text {and} \ \hat{t}_{j}^{x,A} =t_{j'}^{x,A}, \end{aligned}$$

(13)

for $j\in U$, where $j'=\max \{0,1,\dotsc ,j-1\} \backslash U$. In other words, we use the trust rating from the most recent interactions to approximate the missing values. We note that the index $j'$ is well defined in Eq. (13) since we assume the initial trust ratings are known. Now, we can compute $\alpha _{k}^{x,A}$ and $\beta _{k}^{x,A}$ by the approximated trust values as follows

$$\begin{aligned} \begin{aligned} \alpha _{k}^{x,A} =&\alpha _{0}^{x,A} +s^{x,A}\sum _{j\in D_{k}} p_{j}^{A} +\hat{s}\sum _{j\in \overline{D}_{k}}\hat{t}_{j}^{x,y}\left[ \hat{t}_{j}^{y,A} -\hat{t}_{j-1}^{x,A}\right] ^{+}\\ \beta _{k}^{x,A} =&\beta _{0}^{x,A} +f^{x,A}\sum _{j\in D_{k}}\overline{p}_{j}^{A} +\hat{f}\sum _{j\in \overline{D}_{k}}\hat{t}_{j}^{x,y}\left[ \hat{t}_{j-1}^{x,A} -\hat{t}_{j}^{y,A}\right] ^{+} \end{aligned}.\nonumber \\ \end{aligned}$$

(14)

Similar to maximizing H, we can apply the gradient descent method to find the maximizer $\varvec{\theta } _{*}^{x,A}$ of $H_{U}$. The gradient $\nabla H_{U}$ can be computed in the same way as Eq. (10) except that the summation is over $\{0,\dotsc ,K\} \backslash U$ instead of $\{0,\dotsc ,K\}$, i.e.,

$$\begin{aligned} \nabla H_{U}( \varvec{\theta } ^{x,A}) =\sum _{k\in \{0,\dotsc ,K\} \backslash U} {\textbf {C}}_{k} {\textbf {v}}_{k}, \end{aligned}$$

where ${\textbf {C}}_{k}$ and ${\textbf {v}}_{k}$ are defined in Eqs. (11) and (12) and computed with the estimated trust values.

By substituting $\varvec{\theta } _{*}^{x,A}$ to Eq. (14), we can approximate the experiences and further estimate the missing trust rating $t_{u}^{x,A}$ by the expectation $\mu _{u}^{x,A} =\frac{\alpha _{u}^{x,A}}{\alpha _{u}^{x,A} +\beta _{u}^{x,A}}$ for $u\in U$.

4 Human subject study

We conducted a human-subject experiment with 30 participants to evaluate the TIP model. The experiment, inspired by Yang et al. (2017), simulated a search and detection task where two human agents work with two smart drones to search for threats at multiple sites. We examined the participants’ trust dynamics when teaming with the drones.

4.1 Participants

A total of ${N=30}$ participants (average age = 25.3 years, SD = 4.3 years, 16 females, 14 males) with normal or corrected-to-normal vision formed 15 teams and participated in the experiment. Each participant received a base payment of $15 and a bonus of up to $10 depending on their team performance.

4.2 Experimental task and design

In the experiment, pairs of participants performed a simulated threat detection task, aided by two assistant drones, across $K=15$ sessions.¹ They performed the task on two distinct desktop computers. The primary objective of the task was to identify potential threats, which appeared as enemy combatants in screenshots taken from the video game Counter-Strike. The participants were asked to click buttons to indicate whether there were threats on the screen. The drones’ role was to assist the participants in identifying these threats by highlighting them on the screen. To see how participants’ trust in the drones would change, we deliberately programmed the drones to occasionally err, either by signaling false alarms or overlooking actual threats. Since the ground truth regarding the presence of threats was known, we used the Wizard of Oz method to control the drones’ reliabilities.

At each session, each participant was assigned one drone and worked on the detection tasks. After the session, they were asked to report their trust in each drone and their trust in their human teammate. For clarity, we named the two drones A and B and colored them in red and blue, respectively; and we denoted the participants as x and y. A trust rating is denoted as $t^{a,b}_k$, where the superscript $a\in \{x,y\}$ stands for the trustor, the superscript $b\in \{x,y,A,B\}$ stands for the trustee, and the subscript k is the session index. For example, $t^{x,A}_2$ is person x’s trust in drone A after the 2nd session. The range of a trust rating is [0, 1], where 0 stands for “(do) not trust at all” and 1 stands for “trust completely”. The flow of the experimental task is illustrated in Fig. 3a. Here we define each session as an episode of interaction (cf. Section 3.1). An interaction between a human and a drone occurs when the human observes the drone’s performance directly or receives his or her teammate’s trust in the drone indirectly. As a result, after each session, a human had an interaction with his or her drone through direct experience and an interaction with the other drone through indirect experience.

Fig. 3

Experimental process and task interface (Color figure online)

Initial trust rating. Before the detection task, each participant gave their initial trust in the two drones based on their prior experience with automation/robots. Additionally, they gave their initial trust in each other. The pre-interaction value presents their propensity to trust each other and the robots. These trust ratings were indexed by 0, e.g., x’s initial trust rating in A was denoted as $t^{x,A}_0$.

Robot assignment. At each session, each participant was randomly assigned one drone as his or her assistant robot, as shown in Fig. 4.

Fig. 4

Illustration of drone assignment. Participant x is randomly assigned to work with drone A in session 1, with drone B in session 2, and so on. The assignment is random (Color figure online)

Search and detection task. Each session consisted of 10 locations to detect. As shown in Fig. 3b, four views were present at each location. If a threat, which appeared like a combatant, was in any of the views, the participant should click the ‘Danger’ button; otherwise, they should click the ‘Clear’ button. Meanwhile, his or her drone would assist by highlighting a view if the drone detected a threat there. In addition, a 3-second timer was set for each location. If a participant did not click either button before the timer counted down to zero, the testbed would move to the next location automatically. After all the 10 locations, an end-of-session screen was shown, displaying how many correct choices the participant and the drone had made in the current session. Correct choices mean correctly identifying threats or declaring ‘Clear’ within 3 s.

Trust rating. After each session, each participant reported three trust values. First, each participant updated his or her trust in the drone s/he just worked with, i.e., through direct experience, based on the drone’s detection ability. Next, through a server (see Fig. 4), each participant communicated their trust in the drone s/he just worked with to their human teammate. After that, each participant updated his or her trust in the other player’s drone (i.e., through indirect experience). Note that only trust ratings were communicated and drones’ performances were not. Finally, as we need the between-human trust as a discount factor, each participant updated his or her trust in the human teammate based on the teammate’s ability to rate trust in the drones accurately. Hence, after the kth session, there would be 6 additional self-reported trust values, $t^{x,A}_k$, $t^{x,B}_k$, $t^{y,A}_k$, $t^{y,B}_k$, $t^{x,y}_k$, and $t^{y,x}_k$. An illustration of the rating interface is shown in Fig. 3c. After participants completed all 15 sessions, the experiment ended.

4.3 Experimental procedure

Participants were not allowed to interact with each other or with the testbed prior to the experiment. In the beginning of the experiment, each participant signed a consent form and filled in a demographic survey. To familiarize themselves with the setup, two practice sessions were provided, wherein a practice drone was used to assist the participants. The participants were informed that the practice drone differed from the two drones used in the real experiment. We instructed the participants to rate trust in the drones based on how the drones perform the detection task, and their trust in the human teammate based on the teammate’s ability to rate trust in the drones accurately. After the experiment started, the assignment of drones was randomized for each pair of participants. For drone A (B), the detection accuracy was set to 90% (60%) and the number of correct detections in a session followed a binomial distribution B(10, 0.9) (B(10, 0.6)). 90% and 60% were chosen because they are commonly used in Human Factors/HCI studies as the upper and lower bounds of imperfect autonomy (Wickens and Dixon, 2007).

To motivate participants to provide accurate trust ratings, team performance instead of individual performance was used to determine the bonus, which was calculated as $\$10\times \max \{0,(\bar{a}-0.7)/0.3\}$, where $\bar{a}$ was the average detection accuracy of the two participants over all the tasks. Specifically, the participants would receive a bonus if their average detection accuracy exceeded $70\%$. Participants were explicitly informed that truthful and accurate communication of their trust values would assist the other participant in determining the appropriate level of trust in the drones, thereby increasing their detection accuracy and potential bonus.

5 Results and discussion

In this section, we visualize and analyze the data collected from the experiment. We found that trust did propagate between participants within a team and demonstrated the effectiveness of the proposed model in estimating trust in a multi-human-multi-robot team.

5.1 Trust dynamics

We visualize the trust ratings of participants to investigate if trust propagation existed in the experiment. We denote the set of participants as $P=\{x_1,y_1,\dots ,x_{15},y_{15}\}$, where $x_i$ and $y_i$ are two members in the ith team.

We use the trust ratings of participants $x_1$ and $y_1$ as an example to examine the trust dynamics in the experiment, as shown in Fig. 5. The red curve is the trust ratings in the red drone, while the blue curve is in the blue drone. The shaded region indicates the assignment: blue (red) regions indicate that the participant was working directly with the blue (red) drone. To illustrate the effect of indirect experience, we plot small triangles at the sessions where trust was updated via indirect experience. An upward (downward) triangle indicates the current trust from a participant’s teammate in a drone is higher (lower) than the participant’s previous trust in the drone. Moreover, if the actual trust change of a participant has a different direction compared with the direction suggested by his or her teammate, then the triangle is colored in black. For example, in the 1st session, participant $x_1$ worked with the red drone and $y_1$ with the blue drone. $x_1$’s trust in the red drone was higher than $y_1$’s previous trust in the red drone, thus the triangle at $y_1$’s trust in the red drone at the 1st session points upward. At session 4, although $x_1$’s trust in the red drone was higher than $y_1$’s previous trust in the red drone (at task 3), $y_1$ lowered his trust at session 4, so the triangle at task 4 on the $y_1$’s trust in the red drone is in black, indicating the disagreement between $x_1$’s trust and $y_1$’s updated trust. Figure 6 includes the trust ratings of all the participants in the data set. We can see that there are only a few black triangles in the figure, which implies that trust did propagate between two participants within a team during most sessions.

Fig. 5

Trust dynamics of participants $x_1$ and $y_1$. A detailed description is in Sect. 5.1 (Color figure online)

Fig. 6

Trust dynamics of all participants. Black triangles denote the sessions where a participant’s actual change of trust has a different direction compared with the direction suggested by his or her teammate. A detailed description of the legend can be found in Sect. 5.1 (Color figure online)

5.2 Performance and reaction time

To investigate how trust affects a participant’s performance in the detection task and his or her reliance on the drone, we analyzed the performance and reaction time (RT) data.

The performance of a participant at a certain session is defined as the number of correct choices he or she made divided by the total number of detection tasks in a session. We compared the performance and RT between drones A and B using repeated measures ANOVA. We found no significant difference in performance (performance with drone A is 99%, with drone B is 98%), largely due to ceiling effects, but the RT in drone A is significantly shorter than drone B ($\text {RT}_A = 1.59 s$, $\text {SD} = 0.2$; $\text {RT}_B = 1.67 s$, $SD = 0.19$; $p <.01$).

We conducted a more detailed analysis of how trust influenced RT using mixed-effect regression. We found increasing trust reduces reaction time ($\beta = -0.23$, $p <.01$) regardless of drone type. Moreover, we found a significant interaction effect between drone type and trust ($\beta = -0.38$, $p <.01$): When using drone A, increasing trust reduces RT even more (i.e. a steeper slope). This indicates when working with drone A, participants rely more on the drone’s auto-detection and trust has a larger effect on RT ($\beta = -0.61=-0.23-0.38$); when working with drone B, participants rely more on their manual checking and trust has a smaller effect on RT.

5.3 Trust convergence within teams

We conduct two types of team-level analysis to demonstrate that leveraging both direct and indirect interaction with a robot leads to faster trust convergence at the team level. We then compare the with- vs. between-team trust deviation and illustrate statistically the existence and benefits of leveraging both direct and indirect experience for trust updating.

Within-team trust average over time. We calculate the within-team trust average for team i on drone R at session k as

$$\begin{aligned} {t}^{i, R}_k:=\frac{1}{2} \left( t_{k}^{x_{i},R} + t_{k}^{y_{i},R}\right) , \end{aligned}$$

where ${R\in \{A,B\}}$ indicates the drone type. The within-team trust average represents a team of players’ overall trust in a robot.

Fig. 7

Within-team trust average in drones A and B over time. Solid lines indicate mean values and the shaded regions indicate the sample standard errors (Color figure online)

Figure 7 shows how the within-team average trust changed as the number of interactions increased. The initial and final trusts in drone A ($\frac{1}{15} \sum _{i=1}^{15} {t}^{i, A}_0$ and $\frac{1}{15} \sum _{i=1}^{15} {t}^{i, A}_{15}$) were $0.57 \pm 0.16$ (mean ± SD) and ${0.83 \pm 0.09}$, respectively. The initial and final trusts in drone B ($\frac{1}{15} \sum _{i=1}^{15} {t}^{i, B}_0$ and $\frac{1}{15} \sum _{i=1}^{15} {t}^{i, B}_{15}$) were ${0.61 \pm 0.15}$ and $0.46 \pm 0.19$, respectively. A two-way repeated measures analysis of variance (ANOVA) showed a significant main effect of drone type (drone A vs. B, ${F(1,14)=58.81}$, ${p<.001}$), and a non-significant effect of time (initial vs. final, ${F(1,14)= 3.66}$, ${p=.08}$). There was also a significant interaction effect (${F(1, 14)=73.02}$, ${p<.001}$). Prior to the experiment, the within-team average trust in drone A and that in drone B were similar. As the amount of interaction increased, the within-team average trust in drones A and B tended to reflect the different detection accuracy of drone A and drone B, which were set to 90% and 60%, respectively. The within-team average trust in drone A gradually increased and that in drone B decreased. At the end of the experiment, the within-team average trust in drone A was significantly larger than that in drone B (${p<0.001}$).

Within-team trust deviation over time. We define the within-team trust deviation of team i on drone R at session k as the difference in trust ratings between the two human players in a team, regardless of whether the trust update is due to direct or indirect interaction, calculated as

$$\begin{aligned} \text {dev}^{i, R}_{k,\text {W/N}}:=|t_{k}^{x_{i},R} - t_{k}^{y_{i},R} |, \end{aligned}$$

where $R\in \{A,B\}$ is the drone type and the subscript “$\text {W/N}$” stands for “within.” In contrast to the within-team trust average, the within-team trust deviation focuses on the differences between the two players in a team.

Fig. 8

Within-team trust deviation in drones A and B over time. Solid lines indicate the mean values and the shaded regions indicate the sample standard errors (Color figure online)

Figure 8 plots the within-team trust deviation in drone A and drone B. For both drones, the within-team trust deviation decreased rapidly in the first few sessions and became relatively stable afterward. For drone A, the initial and final within-team trust deviations ($\frac{1}{15} \sum _{i=1}^{15} \text {dev}^{i, A}_{0,\text {W/N}}$ and $\frac{1}{15} \sum _{i=1}^{15} \text {dev}^{i, A}_{15,\text {W/N}}$) were ${0.27 \pm 0.25}$ and ${0.06\pm 0.08}$. For drone B, the initial and final trust deviation values ($\frac{1}{15} \sum _{i=1}^{15} \text {dev}^{i, B}_{0,\text {W/N}}$ and $\frac{1}{15} \sum _{i=1}^{15} \text {dev}^{i, B}_{15,\text {W/N}}$) were $0.27 \pm 0.24$ and $0.07 \pm 0.09$. A two-way repeated measures ANOVA revealed a significant main effect of time, that the within-team trust deviation at the end of the experiment was significantly smaller than that prior to the experiment (${F(1,14)=11.51}$, ${p=.004}$). Neither the drone type (${F(1,14)=.06}$, ${p=.82}$) nor the interaction effect (${F(1,14)=.313}$, ${p=.59}$) was significant.

Within- vs. between-team trust deviation. To statistically show the existence of trust propagation among team members, we compare the within-team and between-teams trust deviations as human agents gain more interaction experience. If trust propagation between the two players in a team had not occurred (i.e., participants updated their trust in the drones based solely on direct interaction), the within-team and between-team trust deviation would be statistically equal throughout the entire experiment. The between-team trust deviation of the ith team on drone R after the kth session is defined as

$$\begin{aligned} \begin{aligned}&\text {dev}_{k,\text {BTW}}^{i,R}\\:=&{\frac{1}{N-2}\sum _{p\in P\backslash \{x_{i},y_{i} \}}\frac{1}{2}\left( |t_{k}^{x_{i},R} -t_{k}^{p,R} |+|t_{k}^{y_{i},R} -t_{k}^{p,R} |\right) }, \end{aligned} \end{aligned}$$

where $R\in \{A,B\}$ and N was the total number of participants. Figure 9 illustrates the calculation of within- and between-team trust deviations.

Fig. 9

Within-team trust deviation of team i is the trust difference between $x_i$ and $y_i$, indicated by the dashed line in the Figure. Between-team trust deviation of team i is the average trust difference between the trust ratings of $x_i$ and $y_i$ and all the other participants in other teams, indicated by the solid lines

Figure 10 shows the within- vs. between-team trust deviations at the beginning and end of the experiment. In the beginning, the within- and between-team trust deviations in drone A were $0.28 \pm 0.25$ and $0.27 \pm 0.22$, respectively, and in drone B were $0.27 \pm 0.24$ and $0.25 \pm 0.21$, respectively (Fig. 10a). A two-way repeated measures ANOVA showed no significant difference between the within- and between-team trust deviation (${F(1, 14)=.07}$, ${p=.90}$). No difference was found between drone A and drone B (${F(1,14)=2.82}$, ${p=.12}$). The interaction effect was not significant either (${F(1, 14)=0.75}$, ${p=.40}$).

Fig. 10

Within- vs. between-team trust deviations a at the beginning and b end of the experiment (Color figure online)

Fig. 11

Fitting results. Red curves are for drone A while blue curves are for drone B. The solid lines are the participants’ trust feedback, while the dashed lines are the expected trust value given by the model. The shaded areas indicate the 90% probability interval of the Beta distribution at each session. The index i-j stands for the jth participant in the ithe group. The horizontal axes represent the session number, ranging from 0 (prior interaction index) to 15. The vertical axes indicate trust levels, ranging from 0 to 1, where 0 represents “(do) not trust at all” and 1 indicates “trust completely” (Color figure online)

At the end of the experiment, the within-team and between-team trust deviations in drone A were $0.06 \pm 0.08$ and $0.11 \pm 0.04$, and in drone B were $0.07 \pm 0.09$ and $0.22 \pm 0.08$ (Fig. 10b). A two-way repeated measures ANOVA revealed that the within-team trust deviation is significantly smaller than the between-team deviation (${F(1, 14)=71.16}$, ${p<.001}$), and trust deviation in drone A is significantly smaller than drone B (${F(1, 14)=9.81}$, ${p=.007}$). In addition, there was also a significant interaction effect (${F(1, 14)=5.86}$, ${p=.03}$).

The above results demonstrate the existence, and more importantly, the benefits of trust propagation. As shown in Figs. 7 and 8, the within-team trust average quickly stabilized and the within-team trust deviation rapidly decreased because of trust propagation within a team. Statistically speaking, at the beginning of the experiment, the within-team and between-team trust deviation in both drones were not significantly different (see Fig. 10a). At the end of the experiment, the within-team trust deviation was significantly smaller than the between-team trust deviation (see Fig. 10b). Had there not been trust propagation between the two players in a team (i.e., participants update their trust in the drones based only on the direct interaction), the within-team and between-team trust deviations would remain statistically equal. Therefore, the significant difference at the end of the experiment was attributed to the trust propagation within a team. Being able to fuse one’s direct and indirect experience, instead of relying solely on the direct experience, contributes to the quick convergence of trust assessments on a robot, leading to a significantly smaller within-team trust deviation compared to the between-team trust deviation.

Fig. 12

Fitting error comparison between the TIP model and the two baseline models

5.4 Model fitting

To simplify the notation, we relabel the participants as $P=\{p_1,p_2,\dots , p_{30}\}$. We utilize the gradient descent method in Sect. 3.5 to compute the optimal parameters ${\varvec{\theta }^{p_i,A}_*}$ and ${\varvec{\theta }^{p_i,B}_*}$ for each participant $p_i$. The fitting results are shown in Fig. 11. We set the performance measurements of drone A at session k as ${p_{k}^{A} =A_{k} /10}$ and ${\overline{p}_{k}^{A} =1-p_{k}^{A}}$, where $A_{k}$ is the number of correct choices drone A made in the kth session; and we define $p_{k}^{B}$ and $\overline{p}_{k}^{B}$ similarly. To measure the performance of the model, we define the fitting error at each session for each participant as

$$\begin{aligned} e_{k}^{p_{i},R} =\left| \mu _{k}^{p_{i},R} -t_{k}^{p_{i},R}\right| ,\ R\in \{A,B\}, \end{aligned}$$

where $t_{k}^{p_{i},R}$ is the participant’s reported trust while $\mu _{k}^{p_{i},R}$ is the expected trust computed according to Eq. (2) with $\alpha _{k}^{p_{i},R}$ and $\beta _{k}^{p_{i},R}$ generated by Eq. (8) based on $\varvec{\theta } _{*}^{p_{i},R}$; and, we define the root-mean-square error (RMSE) between the ground truth and the expected trust value as

$$\begin{aligned} \text {RMSE}^{R} =\left[ \frac{1}{N}\sum _{i=1}^{N}\frac{1}{K+1}\sum _{k=0}^{K}\left( e_{k}^{p_{i},R}\right) ^{2}\right] ^{1/2}, \end{aligned}$$

for ${R\in \{A,B\}}$. The results are ${\text {RMSE}^A=0.057}$ and ${\text {RMSE}^B}{\text {=0.082}}$.

To demonstrate its effectiveness, we compare the TIP model with other models. As reviewed in Sect. 2, there has been no research on how human trust forms in multi-agent human–robot interactions. Existing models on trust updating only consider direct experience, and they cannot be extended to the multi-human multi-robot cases directly. Thus, we consider two baseline models: one accounting for solely direct experience and another solely indirect experience. We choose the Beta model in Sect. 3.3 as the direct-experience-only baseline because it has superior performance over previous methods (Guo and Yang, 2020). This model corresponds to the TIP model with zero unit gains in indirect experience, i.e., ${\hat{s}^{x,A}=\hat{f}^{x,A}=0}$; on the other hand, the indirect-experience-only model corresponds to ${{s}^{a,b}={f}^{a,b}=0}$. We recompute the parameters for the baseline models, and the RMSE errors are ${\text {RMSE}^A_{\text {direct}}=0.085}$, ${\text {RMSE}^B_{\text {direct}}=0.107}$, ${\text {RMSE}^A_{\text {indirect}}=0.128}$, and ${\text {RMSE}^B_{\text {indirect}}=0.130}$. In addition, we compare each participant’s fitting error $\bar{e}^{p_{i},R}:={1}/({K+1})\sum _{k=0}^K e_{k}^{p_{i},R}$ of the TIP model (A: $0.044 \pm 0.037$; B: $0.069 \pm 0.045$), direct-experience-only model (A: $0.075 \pm 0.041$; B: $0.095 \pm 0.051$), and indirect-experience-only model (A: $0.116 \pm 0.053$; B: $0.118 \pm 0.054$) using a paired-sample t-test. Results reveal that the fitting error of the TIP model is significantly smaller than the direct-experience-only model, with $t(29)=-6.18, p<.001$ for drone A, and ${t(29)=-7.31}$, ${p<.001}$ for drone B, and significantly smaller than the indirect-experience-only model, with $t(29)=-9.28, p<.001$ for drone A, and ${t(29)=-10.06}$, ${p<.001}$ for drone B. Furthermore, the fitting error of the direct-only model is significantly smaller than the indirect-experience-only model, with $t(29)=-4.73, p<.001$ for drone A, and ${t(29)=-3.73}$, ${p<.001}$ for drone B. A bar plot is shown in Fig. 12. This comparison indicates that a human agent mainly relies on direct experience to update his or her trust, while indirect experience also plays a vital role in trust dynamics.

5.5 Trust estimation

To measure the estimation accuracy of the proposed model, we remove some trust ratings in the data and compute the RMSE of the estimated trust values. Specifically, for each participant $p_{i}$, we set ${U_{\hat{K}} =\{{K-\hat{K} +1},\dotsc ,K\}}$ to remove the last $\hat{K}$ trust ratings, where $U_{\hat{K}}$ is the index set of sessions without trust ratings as defined in Sect. 3.6, and estimate the missing trust values by $\mu _{u}^{p_{i},A}$ and $\mu _{u}^{p_{i},B}$ for $u\in U_{\hat{K}}$. The root-mean-square errors are defined as

$$\begin{aligned} \text {RMSE}_{\hat{K}}^{R} =\left[ \frac{1}{N}\sum _{i=1}^{N}\frac{1}{\hat{K}}\sum _{u=K-\hat{K} +1}^{K}\left( e_{u}^{p_{i},R}\right) ^{2}\right] ^{\frac{1}{2}}, \end{aligned}$$

for $R\in \{A,B\}$.

Figure 13 shows the RMSE’s under different $\hat{K}$. When ${\hat{K}\le 7}$, the TIP model can successfully estimate the trust values in the late sessions with a small RMSE ($< 0.1$) by learning from previous data. In particular, ${\text {RMSE}_{\hat{K} =7}^{A} =0.052}$ and ${\text {RMSE}_{\hat{K} =7}^{B} =0.077}$, which implies that, with the first 9 sessions’ trust ratings available, the RMSE’s of the estimation for the last 7 sessions are under 0.08 for both drones. The result also illustrates that $\text {RMSE}_{\hat{K}}^{A}$ is smaller than $\text {RMSE}_{\hat{K}}^{B}$ in general. This could be explained by the performance difference between the two drones. Indeed, because the number of correct choices each drone could make follows binomial distributions (${\text {Bin}}(10, 0.9)$ for A and ${\text {Bin}}(10, 0.6)$ for B), the variance of their performance are 0.09 and 0.24 respectively. The greater variance of drone B may cause a human subject to acquire more information to stabilize his or her trust and thus leads to higher uncertainty in trust feedback values, which makes it difficult for the model to learn trust dynamics in a short time.

Fig. 13

RMSE’s under different $\hat{K}$. $\hat{K}$ is the number of interactions without trust feedback. Equivalently, the TIP model uses the trust ratings from the first ${16-\hat{K}}$ sessions to learn the model parameters (Color figure online)

6 Conclusion

In the study, we proposed the TIP model that accounts for both the direct and indirect experiences a human agent may have with a robot in multi-human multi-robot teams. To the best of our knowledge, it is the first mathematical framework for computational trust modeling in multi-human multi-robot teams. In addition, we prove theoretically that trust converges after repeated direct and indirect interactions under our TIP framework. Using a human-subject experiment, we showed that being able to fuse one’s direct and indirect experiences, instead of relying solely on the direct experience, contributes to the quick convergence of trust in a robot. In addition, we showed that the TIP models significantly outperformed the baseline direct-experience-only model in capturing the trust dynamics in multi-human multi-robot teams. The TIP model can be applied to various human–robot teaming contexts including team of teams (McChrystal et al., 2015) and multi-echelon networks (National Academies of Sciences, Engineering, and Medicine, 2022). In particular, the TIP model can update a human agent’s trust in a robot whenever a direct or indirect experience is available and thus can be applied for trust estimation in a network consisting of multiple humans and robots.

Our results should be viewed in light of several limitations. First, we assume that the two human players within a team were cooperative and willing to share their trust in a robot truthfully. In a non-cooperative context where a human player is motivated to cheat, a quick convergence of trust assessment is less likely to occur. Further research is needed to examine the non-cooperative scenario. Second, we used a one-dimensional trust scale in the experiment. Even though this scale has been used in prior literature (Manzey et al., 2012; Yang et al., 2017; Bhat et al., 2022), it may not capture the different underlying dimensions of trust. Third, we take an ability/performance-centric view of trust and assume a human agent’s trust in a robot is primarily driven by the ability or performance of the robot. Based on research in organizational management, trust can be influenced by three elements, namely, ability, integrity, and benevolence (Mayer et al., 1995). Future research should investigate ways to integrate the benevolence and integrity elements into trust modeling, in particular, for HRI contexts that involve a strong emotional component, for example, educational or home-care robots. Adding such components will bring new challenges in modeling trust. For example, performance-based trust allows humans to update and self-regulate their trust through repeated interaction with robots. A participant might initially overtrust a robot but adjust the trust level based on the robot’s performance in the following sessions. More complex forms of trust dynamics could be introduced with integrity and benevolence. Also, it is worth investigating the difference between humans’ benevolence and robots’ benevolence (Sica and Sætra, 2023; Buechner et al., 2014), as prior studies assume automation lacks intentionality (Lee and See, 2004). Moreover, conducting further ablation studies is essential to comprehensively understand the impact of various factors on the dynamics of trust. For instance, varying the number of sessions versus drone performances would provide insights into the rate at which trust converges. Finally, our model focuses on the “team of teams” configuration since we implicitly assume that a human can only engage with one robot during a single interaction. Therefore, future studies can explore the scenario where a human can interact with more than one robot simultaneously, which may require a richer representation of interaction.

Appendix A Technical details

1.1 A.1 Proof of Theorem 1

Case 1: $n=0$.

First, we show that $t_{k}^{x,A}$ converges i.p.. When $n=0$, x gains experience towards A through only direct interaction, and thus, by Eq. (3),

$$\begin{aligned} \begin{aligned}&\lim _{k\rightarrow \infty } \mu _{k}^{x,A} =\lim _{k\rightarrow \infty }\frac{\alpha _{k}^{x,A}}{\alpha _{k}^{x,A} +\beta _{k}^{x,A}}\\ =&\lim _{k\rightarrow \infty }\frac{\alpha _{0}^{x,A} +ks^{x,A} r}{\alpha _{0}^{x,A} +\beta _{0}^{x,A} +kf^{x,A}\overline{r} +ks^{x,A} r}\\ =&\frac{s^{x,A} r}{f^{x,A}\overline{r} +s^{x,A} r}. \end{aligned} \end{aligned}$$

(A1)

For any $\epsilon >0$, by the Markov inequality,

$$\begin{aligned} \begin{aligned}&\lim _{k\rightarrow \infty }\Pr (| t_{k}^{x,A} -\mu _{k}^{x,A}| < \epsilon )\\ \leqslant&\lim _{k\rightarrow \infty }\frac{1}{\epsilon ^{2}}\mathbb {E}[( t_{k}^{x,A} -\mu _{k}^{x,A})^{2}] =0, \end{aligned} \end{aligned}$$

(A2)

where the last equality is true because $\lim _{k\rightarrow \infty }{\text {var}}( t_{k}^{x,A}) =0$ as $\lim _{k\rightarrow \infty }( \alpha _{k}^{x,A} +\beta _{k}^{x,A}) =\infty$. Let $t^{x} =\frac{s^{x,A} r}{f^{x,A}\overline{r} +s^{x,A} r}$. Equations (A1) and (A2) yield

$$\begin{aligned} \lim _{k\rightarrow \infty }\Pr (| t_{k}^{x,A} -t^{x}| < \epsilon ) =1. \end{aligned}$$

Second, we show $| t_{k-1}^{y,A} -t_{k}^{x,A}|$ converge i.p. to zero. Suppose this is not true. Then, there exist $\epsilon ,\ \delta >0$ such that there are infinite many k’s such that $\Pr \left( | t_{k-1}^{y,A} -t_{k}^{x,A}|>\epsilon \right) >\delta$. The indirect updating rule Eq. (4) implies $\alpha _{k}^{y,A} +\beta _{k}^{y,A}\xrightarrow {\text {i.p.}} \infty$. Consequently, similar to Eq. (A2), we obtain $| t_{k}^{y,A} -\mu _{k}^{y,A}| \xrightarrow {\text {i.p.}} 0$. We consider the following equation:

$$\begin{aligned} \begin{aligned}&\left| \mu _{k+m-1}^{y,A} -t_{k+m}^{x,A}\right| \\ =&\left| \mu _{k-1}^{y,A} -t_{k}^{x,A}\right| \prod _{j=1}^{m}\frac{\left| \mu _{k+j-1}^{y,A} -t_{k+j}^{x,A}\right| }{\left| \mu _{k+j-2}^{y,A} -t_{k+j-1}^{x,A}\right| }. \end{aligned} \end{aligned}$$

(A3)

As we have shown $t_{k}^{x,A}\xrightarrow {\text {i.p.}} t^{x}$ when $k\rightarrow \infty$, by the continuous mapping theorem, when $k\rightarrow \infty$,

$$\begin{aligned} \prod _{j=1}^{m}\frac{\left| \mu _{k+j-1}^{y,A} -t_{k+j}^{x,A}\right| }{\left| \mu _{k+j-2}^{y,A} -t_{k+j-1}^{x,A}\right| }\xrightarrow {\text {i.p.}}\prod _{j=1}^{m}\frac{\left| \mu _{k+j-1}^{y,A} -t^{x}\right| }{\left| \mu _{k+j-2}^{y,A} -t^{x}\right| }. \end{aligned}$$

(A4)

An examination of Eqs. (2) and (4) shows

$$\begin{aligned} \prod _{j=1}^{m}\frac{\left| \mu _{k+j-1}^{y,A} -t^{x}\right| }{\left| \mu _{k+j-2}^{y,A} -t^{x}\right| }\xrightarrow {\text {i.p.}} 0 \end{aligned}$$

(A5)

when $m\rightarrow \infty$. Equations (A3), (A4), and (A5) together yield $| \mu _{k+m-1}^{y,A} -t_{k+m}^{x,A}| \xrightarrow {\text {i.p.}} 0$ when both k and m tend to infinity. Thus, $\mu _{k}^{y,A}\xrightarrow {\text {i.p.}} t^{x}$. Because we also showed $\left| t_{k}^{y,A} -\mu _{k}^{y,A}\right| \xrightarrow {\text {i.p.}} 0$, $\left| t_{k}^{y,A} -t^{x}\right| \xrightarrow {\text {i.p.}} 0$. This contradicts our assumption. Therefore, $\left| t_{k-1}^{y,A} -t_{k}^{x,A}\right|$ converge i.p. to zero. Particularly, since $t_{k}^{x,A}\xrightarrow {\text {i.p.}} t^{x}$, $t_{k}^{y,A}\xrightarrow {\text {i.p.}} t^{x}$.

Therefore, both $t_{k}^{x,A}$ and $t_{k}^{y,A}$ converge to $t^{x}$ i.p..

Case 2: $n >0$.

First, when $k\rightarrow \infty$, $\alpha _{k}^{x,A}$, $\beta _{k}^{x,A}$, $\alpha _{k}^{y,A}$, and $\beta _{k}^{y,A}$ all go to infinity as both x and y will have an infinite number of direct interactions with A. As a result, we have

$$\begin{aligned} \begin{aligned} \left| t_{k}^{x,A} -\mu _{k}^{x,A}\right|&\xrightarrow {\text {i.p.}} 0&\text {and}\\ \left| t_{k}^{y,A} -\mu _{k}^{y,A}\right|&\xrightarrow {\text {i.p.}} 0.&\end{aligned} \end{aligned}$$

(A6)

Second, let $\Delta t_{k}:=t_{k}^{x,A} -t_{k}^{y,A}$. With the similar technique used in Eqs. (A3), (A4), and (A5), it can be shown that

$$\begin{aligned} \Delta t_{k}\xrightarrow {\text {i.p.}} \Delta t, \end{aligned}$$

(A7)

where $\Delta t$ is some constant.

Finally, we show $t_{k}^{x,A}$ and $t_{k}^{y,A}$ converge to constants i.p.. Since $\left| \mu _{k}^{x,A} -\mu _{k-1}^{x,A}\right| \xrightarrow {\text {i.p.}} 0$, by Eqs. (A6) and (A7), we obtain

$$\begin{aligned} \left| t_{k}^{y,A} -\mu _{k-1}^{x,A}\right| \xrightarrow {\text {i.p.}} \Delta t. \end{aligned}$$

By Eqs. (3) and (4), both indirect and direct experience gains of y converge to some constants i.p. Therefore, the ratio $\frac{\ \alpha _{k}^{y,A}}{\alpha _{k}^{y,A} +\beta _{k}^{y,A}}$ converge to some constants i.p. respectively, i.e., there exist some $t^{y}$ such that $t_{k}^{y,A}\xrightarrow {\text {i.p.}} t^{y}$. Similarly, there exists some $t^{x}$ such that $t_{k}^{x,A}\xrightarrow {\text {i.p.}} t^{x}$. $\square$

1.2 A.2 Proof of Theorem 2

When $n=0$, Eq. (5) yields $t^{x} =t^{y} =\frac{s^{x,A} r}{f^{x,A}\overline{r} +s^{x,A} r}$, which agrees with the proof of theorem 1.

Now we consider the case $n >0$. Let $t^{x}$ and $t^{y}$ be the equilibrium in the statement. If $S^{x} /F^{x} -S^{y} /F^{y} \geqslant 0$, it can be shown that

$$\begin{aligned} \lim \limits _{k\rightarrow \infty }\Pr \left( t_{k}^{x,A} -t_{k}^{y,A} \geqslant 0\right) =1. \end{aligned}$$

As a result, we have

$$\begin{aligned} \left[ t_{k-1}^{y,A} -t_{k}^{x,A}\right] ^{+}\xrightarrow {\text {i.p.}} 0\ \text {and} \ \left[ t_{k}^{x,A} -t_{k-1}^{y,A}\right] ^{+}\xrightarrow {\text {i.p.}} t^{x} -t^{y}.\nonumber \\ \end{aligned}$$

(A8)

When $k\rightarrow \infty$, by Eqs. (3), (4), and (A8), we have

$$\begin{aligned} \begin{aligned} \alpha _{kl+l}^{x,A} -\alpha _{kl}^{x,A}&\xrightarrow {\text {i.p.}} S_{A}^{x}\\ \beta _{kl+l}^{x,A} -\beta _{kl}^{x,A}&\xrightarrow {\text {i.p.}}\hat{F}_{A}^{x}\left( t^{x} -t^{y}\right) +F_{A}^{x} \end{aligned}, \end{aligned}$$

where $l=m+n$. This implies the trust gains are constant in every other l interaction. By Eq. (1),

$$\begin{aligned} t^{x} =\lim _{k\rightarrow \infty }\frac{\ \alpha _{k}^{x,A}}{\alpha _{k}^{x,A} +\beta _{k}^{x,A}} =\frac{S_{A}^{x}}{S_{A}^{x} +\hat{F}_{A}^{x}\left( t^{x} -t^{y}\right) +F_{A}^{x}}, \end{aligned}$$

which proves the first equation in Eq. (5). The second equation in Eq. (5) can be proved similarly.

The proof for the case when $S_{A}^{x} /F_{A}^{x} -S_{A}^{y} /F_{A}^{y} < 0$ is similar. $\square$

1.3 A.3 Computing $t^x_A$ and $t^y_A$

We consider the case of Eq. (5). When $n=0$, the solution is given by Eq. (A1). Assume $n\ne 0$. Solving Eq. (5) directly results in two cubic equations of $t^x$ and $t^y$ respectively. An exact solution can be derived from these equations on $[0,1]^2$ but the process can be tedious. A more practical method is to exploit the Newton’s method to approximate $t^x$ and $t^y$. For example, letting $z=1-y$, Eqs. in (5) give

$$\begin{aligned} \begin{aligned} \hat{F}^{x} x^{2} +\hat{F}^{x} zx+\left( F^{x} +S^{x} -\hat{F}^{x}\right) x-S^{x} =&0,&\text {and}\\ \hat{S}^{y} z^{2} +\hat{S}^{y} zx+\left( S^{y} +F^{y} -\hat{S}^{y}\right) z-F^{y} =&0.&\end{aligned} \end{aligned}$$

We can define $f_1(x,z)$ and $f_2(x,z)$ to be the left-hand sides of above equations and define $\textbf{f} =( f_{1},f_{2})$. We have

$$\begin{aligned} \begin{aligned}&\textbf{f} '( x,z)\\&\quad = \begin{pmatrix} 2\hat{F}^{x} x+\hat{F}^{x} z+F^{x} +S^{x} -\hat{F}^{x} & \hat{F}^{x} x\\ \hat{S}^{y} z & 2\hat{S}^{y} z+\hat{S}^{y} x+S^{y} +F^{y} -\hat{S}^{y} \end{pmatrix}. \end{aligned} \end{aligned}$$

We solve

$$\begin{aligned} ( x_{k+1},z_{k+1})^{T} =( x_{k},z_{k})^{T} -(\mathbf {f'})^{-1}\textbf{f}( x_{k},z_{k}) \end{aligned}$$

iteratively for x and z and obtain $t^x=x$ and $t^y =1-z$ as the equilibrium.