A finite characterization of perfect equilibria

Abstract

Govindan and Klumpp [7] provided a characterization of perfect equilibria using Lexicographic Probability Systems (LPSs). Their characterization was essentially finite in that they showed that there exists a finite bound on the number of levels in the LPS, but they did not compute it explicitly. In this note, we draw on two recent developments in Real Algebraic Geometry to obtain a formula for this bound.

1 Introduction

The concept of perfect equilibrium [15] plays a central role in the theory of refinements of Nash equilibria. Not only has it been successful in applications to economic models, but its pioneering use of trembles has spurred further refinements. From a practical viewpoint, the very nature of its definition makes perfect equilibrium very hard to compute, which explains the relative paucity of algorithms to compute it.¹ Indeed, recall that an equilibrium of a finite game in normal form is perfect if there exists a sequence of profiles of completely-mixed strategies converging to it against which the equilibrium is a best reply. The set of perfect equilibria of a finite game is, thus, defined by finitely many polynomial inequalities as well the universal ($\forall$) and existential ($\exists$) quantifiers. Consequently, checking whether a strategy profile is perfect, let alone computing the entire set, requires, in principle, an infinite number of operations, as we have to contend with the quantifiers: for every $\varepsilon > 0$, there exists $\delta > 0$ such that... However, the set of perfect equilibria is a semi-algebraic set—see [3]. Therefore, by the Tarski-Seidenberg Theorem [4], there exists an equivalent description of perfect equilibria that is quantifier-free. It would be really convenient, then, to know what such a description would look like. This question is open and seems hard to resolve. A more modest question is whether we can eliminate the universal quantifier; in other words, we are after an equivalent definition of perfection of the following form: a strategy profile is perfect if there exists a solution to a finite system of polynomial equations and inequalities (in which the given strategy profile is a parameter). Such a formulation provides a finite characterization of perfection, since it requires only a finite number of steps to check whether a finite system of polynomial equations and inequalities has a solution. Govindan and Klumpp [7] (henceforth GK) obtain a result of this kind. The system that they consider is shown to have a bound on the number of polynomials involved; this bound depends only on the cardinalities of the player set and the strategy sets of the players, but it is not computed. In this note, we sharpen the result in GK by giving explicit bounds. It is worth noting that the polynomial system in this characterization involves only multilinear polynomials, which are especially suitable for the application of polynomial-solving algorithms (see [6] and [11]).

What make this paper possible are recent advances in quantifying the bounds involved in two results from Real Algebraic Geometry that were used by GK. The first concerns what is called the Łojasiewicz inequality, which states that the value of a polynomial in a neighborhood of a zero is bounded from below by a polynomial function of the distance from the zero-set of the polynomial. Kurdyka and Spodzieja [10] give an explicit formula for the degree of the bounding polynomial. The second concerns the Nash curve-selection lemma [4]. Suppose X is a semi-algebraic set and x belongs to its closure. Then, the curve-selection lemma states that there is an analytic function from an interval $[0, \varepsilon ]$ into the closure of X that maps 0 to x and maps all other points into X. Basu and Roy [1] provide a quantitative version of this lemma that gives us a bound on the degree of the (coordinate) analytic functions.

2 Definitions and statement of the theorem

We study a finite game G in normal form. The set of players is ${\mathcal {N}}= \{\, 1, \ldots , N \, \}$. The finite set of pure strategies of each player $n \in {\mathcal {N}}$ is denoted $S_n$, and the corresponding set of mixed strategies is $\Sigma _n$. Define $S \equiv \prod _n S_n$ and $\Sigma \equiv \prod _n \Sigma _n$. For each n, $S_{-n} = \prod _{m \ne n} S_m$; $\Sigma _{-n} \equiv \prod _{m \ne n} \Sigma _m$. The payoff function of player n is $G_n: \Sigma \rightarrow {\mathbb {R}}$.

Definition 2.1

The profile $\sigma \in \Sigma$ is a perfect equilibrium if there exists a sequence $\sigma ^k$ of profiles of completely-mixed strategies converging to $\sigma$ such that for each k, each player $n \in {\mathcal {N}}$, and each strategy $s_n \in S_n$, $G_n(\sigma _n, \sigma _{-n}^k) \ge G_n(s_n,\sigma _{-n}^k)$.

GK provide an equivalent definition of perfect equilibria that replaces the test sequence $\sigma ^k$ with a lexicographic probability system (LPS) for each player.² To describe their characterization, we need some definitions.

Definition 2.2

Let K be a non-negative integer. An LPS of order K over a finite set X is a $(K+1)$-tuple $(\varrho ^0, \ldots , \varrho ^K)$ of probability distributions over X. We say that $\varrho$ has full support if $\cup _{k = 0}^{K} \, \text {supp}\varrho ^k = X$.

Definition 2.3

An LPS profile of order K over S is an N-tuple $\varrho \equiv (\varrho _1, \ldots , \varrho _N)$, where for each player n, $\varrho _n$ is an LPS over $S_n$ of order K. The LPS profile $\varrho$ has full support if each $\varrho _n$ has full support, and in this case, let $\ell (\varrho ) \equiv \max _{n \in {\mathcal {N}}} \min \{ \, k \mid \cup _{i = 0}^k \, \text {supp} \varrho _n^i = S_n \, \}$.

The next definition gives us a procedure for forming products of the LPSs of the players.

Definition 2.4

Given an LPS profile $\varrho$ of order K over S, for each player n, the induced beliefs $\mu _n$ over $S_{-n}$ of order $K(N-1)$ is defined as follows. For $k = 0, \ldots , K(N-1)$:

$$\begin{aligned} \mu _n^k = C_n^k\sum _{{(k_m)}_{m \ne n}} \otimes _{m \ne n} \varrho _{m}^{k_m}, \end{aligned}$$

where the sum is over all vectors ${(k_m)}_{m \ne n}$ whose coordinates sum to k, and $C_n^k$ is the appropriate normalizing constant that gives us a probability distribution.

Definition 2.5

Let $\varrho$ be an LPS profile of order K and let $\mu _n$ be the induced beliefs for player n. For $0 \le k \le K(N-1)$, we say that a strategy $\tau _n \in \Sigma _n$ is a best reply of order k against $\varrho$ if for all $s_n \in S_n$:

$$\begin{aligned} (G_n(\tau _n, \mu _n^0), \ldots , G_n(\tau _n, \mu _n^k)) \ge _L (G_n(s_n, \mu _n^0), \ldots , G_n(s_n, \mu _n^k)), \end{aligned}$$

where $\ge _L$ is the lexicographic ordering on vectors.

We are now ready to state the main result of GK.

Theorem 2.6

Given a normal-form game G, there exist non-negative integers $\ell \le K$ (that depend only on N and the cardinalities of the sets $S_n$) such that a strategy profile $\sigma$ is a perfect equilibrium of G iff there exists an LPS profile $\varrho$ of order K such that:

• (1) $\varrho$ has full support and $\ell (\varrho ) \le \ell$;

• (2) $\varrho _n^0 = \sigma _n$ for each player n;

• (3) $\sigma _n$ is a best-reply of order K against $\varrho$.

For two-player games, we can take $\ell = K = 1$. For the general case, GK only proved that these bounds exist. For our bounds, we need to define a few constants. Let $\kappa = \sum _{n \in {\mathcal {N}}}|S_n|$; $d = N-1$; $Z = (2d +6)(2d+5)^{\kappa -1}$; $D = 5(\kappa (2d+4) +2)Z$; ${\mathcal {L}}= d{(6d - 3)}^{2\kappa - 1}$.

Theorem 2.7

In Theorem 2.6, we can take $\ell = 2d(ZD)^2(1+Z)$ and $K = {\mathcal {L}}2d(ZD)^2(1+Z)$.

Remark 2.8

We have defined K to be $\ell {\mathcal {L}}$, but as the proof of Theorem 2.7 shows, we do not have to let K be this absolute constant, but rather we could let it be $\ell (\varrho ){\mathcal {L}}$. Put differently, in the statement of Theorem 2.6, $\sigma$ is perfect iff we have an LPS satisfying conditions (1) and (2) and where $\sigma _n$ is a best-reply of order $\ell (\varrho ){\mathcal {L}}$ against $\varrho$.

Remark 2.9

Suppose we have an N-player game where each player has a actions, then using very crude bounds for Z and D, we get $\ell \le 100N^3a^2{(6aN)}^{6aN} \equiv {{\bar{\ell }}}$ and $K \le {(6N)}^{2aN} {{\bar{\ell }}}$. Of course, these bounds are enormously high, but we believe that they can be improved upon (see the last section of the paper).

Remark 2.10

As a practical matter, what can be said of our characterization when the payoffs are integers? It is well-known that even with integer payoffs, a game may not have an equilibrium with rational coordinates—cf. [13]. Therefore, the best one can hope for is that the LPS characterization we have involves algebraic numbers, and indeed that is the case. To see why that is true, observe that an LPS test for perfection involves obtaining probability distributions (the levels of an LPS) that solve a finite system of polynomial equations and inequalities where the coefficients are the payoff numbers in the game. Thus, all the probability distributions of the LPS have only algebraic numbers.

3 Some facts about polynomials

GK prove their theorem by first deriving an equivalence between the statement using LPSs and another involving polynomials. Applying tools from semi-algebraic geometry for polynomials, they then derive their bounds. The proof of our theorem works with the same set of polynomials and therefore we now review some facts about polynomials and also the above-mentioned equivalence of GK.

A monomial function $F: {\mathbb {R}}^{\kappa } \rightarrow {\mathbb {R}}$ is of the form $x_1^{i_1}\cdots x_k^{i_k}$ where $i_1, \ldots i_k$ are non-negative integers; its degree in variable $x_l$, denoted $\text {deg}_{x_l} (F)$, is $i_l$ and its total degree, denoted $\text {deg} (F)$, is $i_1 + \cdots + i_k$. A polynomial is a finite linear combination $\sum _j a_j F_j$ of mononomials $F_j$; its degree in variable $x_l$ is $\max \, \{\, \text {deg}_{x_l} \, F_j \mid a_j \ne 0 \, \}$ and its total degree is $\max \, \{\, \text {deg} \, F_j \mid a_j \ne 0 \, \}$, with the degree of the zero function being zero. If $F: {\mathbb {R}}^\kappa \rightarrow {\mathbb {R}}^l, l \ge 2$, is a function where each coordinate $F_j$ is a polynomial, the degree of F is the maximum over j of the degree of $F_j$. We call F a polynomial map.

For a polynomial (or more generally a power series) $f(t) \equiv \sum _k a_k t^k$ of a single variable t, the order of f, denoted o(f), is the smallest k for which $a_k \ne 0$—the order of the zero function is $\infty$. We say that $f > 0$ (resp. $f \ge 0)$ if $a_{o(f)} > 0$ (resp. either $a_{o(f)} > 0$ or $f \equiv 0$). For a polynomial map $f: {\mathbb {R}}\rightarrow {\mathbb {R}}^l$, the order of f is $\max _j o(f_j)$.

Now we turn to the description of perfect equilibria using polynomials. A polynomial strategy-profile is a polynomial map $\eta : {\mathbb {R}}\rightarrow \prod _{n \in {\mathcal {N}}} {\mathbb {R}}^{S_n}$. For each n, the payoff function $G_n$ can be extended uniquely to a multilinear function over the whole of $\prod _{m \in {\mathcal {N}}} {\mathbb {R}}^{S_m}$, still denoted $G_n$. Given a polynomial strategy profile, we can now compute the “payoff” $G_n(\eta ) \equiv G_n \circ \eta$, which is a polynomial whose degree is at most $\sum _{m}\max _{s_m} \text {deg} \, \eta _{m,s_m}$. We say that a strategy $\tau _n \in \Sigma _n$ is a best-reply of order r against a polynomial strategy profile $\eta$ if for each $s_n \in S_n$, $G_n(\tau _n, \eta _{-n}) - G_n(s_n, \eta _{-n})$ is either non-negative or of order at least $r+1$. The following lemma is from GK (see their Claim 3.3).

Lemma 3.1

Let $\ell \le K$ be non-negative integers and let $\sigma \in \Sigma$. The following statements are equivalent:

• (1) There exists an LPS profile $\varrho$ of order K such that:

  • (a) $\varrho$ has full support and $\ell (\varrho ) \le \ell$;

  • (b) $\varrho _n^0 = \sigma _n$ for each player n;

  • (c) $\sigma _n$ is a best-reply of order K against $\varrho$.

• (2) There exists a polynomial map $\eta : {\mathbb {R}}\rightarrow {\mathbb {R}}^\kappa$ such that:

  • (a) $\eta _{n,s_n} > 0$ for each $n, s_n$ and $o(\eta ) \le \ell$;

  • (b) $\eta (0) = \sigma$;

  • (c) For each n, $\sigma _n$ is a best reply of order K against $\eta$.

Remark 3.2

As the proof of Lemma 3.1 in GK shows, we can take $o(\eta )$ to be equal to $\ell (\varrho )$ in going from an LPS profile to the associated polynomial and vice versa, thus giving us an intimate connection between these two ways of looking at perfection.

Remark 3.3

There are cases where simple bounds can be obtained for $\ell$ and K using the equivalence of the lemma above. First note that $\sigma$ is a perfect equilibrium if and only if $\sigma$ belongs to the closure of $P \equiv \{\, \tau \in {\mathbb {R}}^{\kappa } \mid \tau \in \text {int}(\Sigma ), \forall n \in {\mathcal {N}}, s_n \in S_n, G_n(\sigma _n, \tau _{-n}) - G_n (s_n, \tau _{-n}) \ge 0 \, \}$. If P is a convex set—as it is the case with two-player or polymatrix games, for example—then consider $\tau ^* \in P$. It follows that the linear map $t \mapsto (1-t)\sigma + t\tau ^*$ satisfies conditions 2(a), 2(b) and 2(c) which implies that $\ell$ and K can be taken equal to 1.

We conclude this section with the two key results from Real Algebraic Geometry that we referred to before. Let $F: {\mathbb {R}}^\kappa \rightarrow {\mathbb {R}}^l$ be a polynomial map of degree d and let V(F) be the set of zeros of F. Fix $x \in V(F)$. The Łojasiewicz inequality provides a lower bound on the value of F in a neighborhood of x. Specifically, there exist positive constants $C, \varepsilon , r$ such that $\Vert F(y) \Vert \ge C \text {dist}{(y, V(F))}^r$ for y such that $\Vert x - y \Vert < \varepsilon$, where $\Vert \cdot \Vert$ is the Euclidean norm and $\text {dist}(y, V(F))$ is the Euclidean distance of y to V(F). The smallest r satisfying the inequality is called the Łojasiewicz exponent. Kurdyka and Spordieza [10] show that if F is a polynomial of degree $d \ge 2$, then the Łojasiewicz exponent is $\le d{(6d-3)}^{\kappa - 1}$. We exploit this estimate in our theorem. The next result concerns the curve-selection lemma.

Proposition 3.4

Let P be a semi-algebraic subset of ${\mathbb {R}}^{\kappa }$ defined by polynomials whose total degrees are bounded by d. Let x belong to the closure of P. There exist $\varepsilon >0$ and an analytic function $\phi : [0,\varepsilon ) \rightarrow {\mathbb {R}}^{\kappa }$ such that: (1) $\phi ((0,\varepsilon )) \subset P$; (2) $\phi (0) = x$; (3) $o(\phi ) \le 2dZ^2D^2(1+Z)$.

Proof

The Quantitative Curve Selection Lemma (Theorem 2 in Basu and Roy [1]) implies that there exist: (1) a semi-algebraic path $\phi : [0, t_0) \rightarrow {\mathbb {R}}^{\kappa }$; (2) a set of polynomials $f(T, U),g_0(T, U),...,g_{\kappa }(T, U)$ in two variables (T, U); (3) a semi-algebraic function $u:[0, t_0) \rightarrow {\mathbb {R}}$ such that: (a) $\phi (0) = x$ and $\phi (t) \in P$ for all $t > 0$; (b) $f(t, u(t)) = 0$ for all $t \in (0, t_0)$; (c) $\phi (t) = (\frac{g_1(t,u(t))}{g_0(t,u(t))},...,\frac{g_{\kappa }(t,u(t))}{g_{0}(t,u(t))})$, for $t>0$.³ Moreover,

$$\begin{aligned} \max \{\text {deg}_{T}(f), deg_{T}(g_0),..., deg_{T}(g_{\kappa })\} = 2dZD^{2} \end{aligned}$$

and

$$\begin{aligned} \max \{\text {deg}_{U}(f), deg_{U}(g_0),..., deg_{U}(g_{\kappa })\} = Z. \end{aligned}$$

Viewing the polynomial f as a polynomial with complex coefficients, it follows from the algebraic closure of complex Puiseux series that the root $u(\cdot )$ can be assumed to be a real Puiseux series. Moreover, by Riemman’s method of resolution of singularities (see Theorem 1.5 in Kollár [9]) we have that u(t) is a real Puiseux series with nonnegative exponents: $u(t) = \sum _{k \ge 0}a_k t^{k/q}$, where $q \le Z$. Riemann’s result also implies that the Puiseux series converges in a neighborhood of zero. Let $\xi$ be the order of u. Now, a necessary condition for (t, u(t)) to be a root of $f(T,U) = \sum _{(i,j)} c_{ij}T^i U^j$ for $t \in [0,t_0)$ is that the lowest powers of t after substituting (t, u(t)) for (T, U) must cancel. Therefore, there must be at least two monomials $c_{ij} T^i U^j$ and $c_{i'j'}T^{i'} U^{j'}$ such that both give the same degree $\beta$ on t after substituting (t, u(t)) in the monomials, and other monomials give degrees $\ge \beta$. Therefore, $i + j\xi = i' + j' \xi \iff \xi = \frac{i - i'}{j' - j} \le 2dZD^2$.

Since $\text {deg}_T(g_{i})$ is bounded by $2dZD^2$ and $\text {deg}_{U}(g_{i})$ is bounded by Z, it follows that the order of $g_{i}(t,u(t))$ is bounded by $(2dZD^2 + 2d(ZD)^2)$. Changing variables from t to $t^q$, it follows that the order of $g_{i}(t^q,u(t^q))$ is less than $(2dZD^2 + 2d(ZD)^2)Z$. Since $\phi$ is continuous at 0, it follows that $\frac{g_{i}(t^q, u(t^q))}{g_{0}(t^q, u(t^q))}, i = 1,...,\kappa$ is a power series with order bounded by $(2dZD^2 + 2d(ZD)^2)Z = (2d(ZD)^2)(1+Z)$.$\square$

4 Proof of Theorem 2.7

As we remarked in Sect. 2, for the case $N = 2$, we can take $\ell = K = 1$. Therefore, assume $N > 2$. We prove the theorem by invoking the equivalence in Lemma 3.1. Fix $\sigma \in \Sigma$. Suppose $\ell$ and K are as specified in Theorem 2.7 and suppose $\eta : {\mathbb {R}}\rightarrow {\mathbb {R}}^k$ is a polynomial map satisfying properties 2(a)–2(c) of Lemma 3.1. We show that $\sigma$ is perfect. There is nothing to prove if $\sigma$ is completely mixed; therefore assume that it is not. For each n, let $T_n$ be the set of pure strategies that are best replies against $\sigma$. Let $T_n^1$ be the subset of $T_n$ consisting of those strategies that are best replies of order K against $\eta$; then $T_n^1$ includes the support of $\sigma _n$. Let $T_n^0 = T_n \backslash T_n^1$. Let $F: {\mathbb {R}}^\kappa \times \prod _n {\mathbb {R}}^{T_n^0} \rightarrow \prod _{n} {\mathbb {R}}^{T_n}$ be the polynomial whose coordinate for $(n, t_n)$ is $F_{n, t_n}(\tau , c) \equiv G_n(t_n, \tau _{-n}) - G_n(\sigma _n, \tau _{-n})$ if $t_n \in T_n^1$ and $F_{n, t_n}(\tau , c) \equiv G_n(t_n, \tau _{-n}) - G_n(\sigma _n, \tau _{-n}) + c_{n,t_n}^2$ if $t_n \in T_n^0$. (In case $T_n^0$ is empty for some n, then we do not have the coordinate c for him; in particular if $T_n^0$ is empty for all n, F is a function defined on ${\mathbb {R}}^\kappa$.) For all small t, each player n and each strategy $t_n \in T_n^0$, $G_n(t_n, \eta _{-n}(t)) - G_n(\sigma _n, \eta _{-n}(t)) <0$. Therefore, the function $c_{n,t_n}(t) \equiv {(G_n(\sigma _n, \eta _{-n}(t) - G_n(t_n, \eta _{-n}(t)))}^{1/2}$ is well-defined. The map $F(\eta (t), c(t))$ now has order $K + 1$ as a function of t.

Let W be the set of $(\tau , c) \in {\mathbb {R}}^{\kappa } \times \prod _n {\mathbb {R}}^{T_n^0}$ such that $\tau _{n,s_n} = 0$ for some n and $s_n$ that is not in the support of $\sigma _n$. We claim that $(V(F) \backslash W) \cap U$ is nonempty for each small neighborhood U of $\sigma$. Indeed, by the Łojasiewicz inequality, if this intersection is empty, then for each small t, since $d((\eta (t), c(t)), V(F)) = O(t^{o(\eta )})$, the order of $F(\eta (t), c(t))$ is no more than $o(\eta ){\mathcal {L}}\le K$, which is a contradiction with the conclusion from the previous paragraph. Hence, the intersection is nonempty.

Take now a sequence of neighborhoods $U_k$ of $\sigma$ whose intersection is $\sigma$. For each k, pick a point $(\sigma ^k, c^k) \in (V(F) \backslash W) \cap U_k$. For each n, all the strategies in $T_n^1$ are equally good replies against $\sigma ^k$, and at least weakly better than those in $T_n^0$; as strategies in $S_n \backslash T_n$ are inferior replies against $\sigma$, they remain so against $\sigma ^k$ for large k. Therefore, $\sigma$ is a perfect equilibrium.

To prove the other direction, let now $\sigma$ be a perfect equilibrium of G. Let P be the set of completely mixed strategy-profiles $\tau \in \Sigma$ such that for all $n \in {\mathcal {N}}, s_n \in S_n$, $G_n(\sigma _n, \tau _{-n}) - G_n (s_n, \tau _{-n}) \ge 0$. The set P is semi-algebraic. Moreover, as $\sigma$ is perfect, P is non-empty and $\sigma$ belongs to the closure of P. By Proposition 3.4, it follows that there exists an analytic function $\varphi : [0,\varepsilon ) \rightarrow {\mathbb {R}}^\kappa$ such that $\varphi ((0,\varepsilon )) \subset P$, $\varphi (0) = \sigma$ and $o(\varphi ) \le \ell$. Therefore, $K \le (2d(ZD)^2)(1+Z){\mathcal {L}}$. Now consider the polynomial $\eta$ of order K obtained by truncating each coordinate of $\varphi$ to its first $K+1$ terms. This polynomial satisfies conditions 2(a)–2(c) of Lemma 3.1 and our theorem is proved.

5 Concluding remarks

The two quantitative results concerning the Łojasiewicz inequality and the curve-selection lemma that we invoke hold for the case of arbitrary polynomials. In the context of game theory, the polynomials we are considering are very special: they happen to be multilinear functions. Therefore, it is worthwhile investigating whether the bounds in this paper can be tightened.

The idea of trembles in the definition of perfection is the basis for a number of refinements, and we can obtain a finite characterization for several of those as well. As extensive-form perfection is the same as normal-form perfection applied to the agent-normal-form, the results here extend immediately to it. As for properness [12], as GK show, there is an equivalent definition using LPSs that is similar to that for perfection with the added restriction that superior replies are infinitely more likely in the LPS. By incorporating an additional variable $\varepsilon$ (used in the definition of $\varepsilon$-properness), we get a finite characterization where the variables $\kappa$ and d are augmented by one. Finally, a more challenging and important open problem is to obtain a similar characterization for stable sets [8].⁴