Economic Reasoning and AI
Economic Reasoning and AI
Economic Reasoning and AI
1
Harvard John A. Paulson School of Engineering and Applied Fig. 1. A bounded reinforcement learning agent performs better by pursuing a designed reward
Sciences, Harvard University, 33 Oxford Street, Cambridge MA function different from the objective reward: its actual fitness evaluation. Results (left) from a
02138, USA. 2Computer Science and Engineering, University of
gridworld foraging domain (right), for various limits on the agent’s planning horizon (84). Unless the
Michigan, 2260 Hayward Street, Ann Arbor, MI 48109, USA.
*Corresponding author. E-mail: [email protected] (D.C.P.); agent is perfectly rational (i.e., no horizon limit)—not typically feasible in realistic applications—the
[email protected] (M.P.W.) designer can often achieve better fitness by directing the agent to optimize an alternative measure.
Fig. 2. Researchers produced steady exponential progress on solving games of imperfect information from 1995 to the present. Up to 2007 (left), game
size was generally reported in terms of nodes in the game tree. Based on methods introduced around that time, it became more meaningful (right) to report size in
terms of the number of information sets (each many nodes), which represent distinct situations as perceived from the perspective of a player. The circled data
We focus here on some of the research direc- standing an agent’s attitudes solely in terms of plex domains, and enabling inference about pre-
tions we consider most salient for a future syn- its internal data structures. Imperfections in ferences) is regarded as a necessary AI facility.
thesis of economics and AI engendered by the decision-making may mean that the beliefs held Planning, the AI subfield concerned with action
emergence of machina economicus. Interesting and objectives pursued by a computational agent, over time, now conventionally frames its prob-
as they are, we only briefly mention here the in effect, vary systematically from those directly lem as one of optimization, subject to resource
many exciting applications of AI to problems in encoded. constraints, multiple objectives, and probabilistic
economics such as matching (9), market clearing As an example illustrating this distinction, effects of actions.
(10), and preference modeling for smart grids machine-learning researchers adapted from ani- Will AI succeed in developing the ideal ration-
(11). Nor will we showcase the many ways in mal learning the concept of reward shaping (14). al agent? As much as we strive to create machina
which economic theory is finding application In reinforcement learning, the agent derives a economicus, absolutely perfect rationality is un-
today within AI—for example, game-theoretic policy (mapping from perception sequences to achievable with finite computational resources.
approaches to multi-agent learning (12) and actions) based on rewards representing instan- A more salient question is whether AI agents will
voting procedures to combine the opinions of taneous value associated with a state and action. be sufficiently close to the ideal as to merit think-
AIs (13). A designer specifying the input reward can often ing about them and interacting with them in
train the agent more efficiently by shaping the rationalistic terms. Such is already the case, at
Building machina economicus reward signal over the learning process to facil- least in a limited sense. Whenever we anthropo-
Constructing a rational AI raises a host of tech- itate convergence to behavior optimizing the morphize our machines, we are essentially treat-
nical challenges not previously addressed in the designer’s objective. The framework of optimal ing them as rational beings, responding to them
long tradition of rationalistic modeling in the rewards (15) provides a general treatment distin- in terms of our models of their knowledge, goals,
social sciences. For economics, the agent atti- guishing reward specifications and designer goals. and intentions. A more refined version of the
tudes (e.g., beliefs and preferences) underlying As shown in Fig. 1, the optimal reward input to question is whether our formal rationality the-
rationality are conceptual abstractions. Econo- the agent does not generally correspond to the ories will fit well the behavior of AI agents in
mists need not explain how capabilities and designer’s ideal reward. This perspective helps absolute terms or compared to how well the
preferences, for example, are encoded, nor the explain the role of intrinsic motivations (e.g., theories work for people. Without offering any
algorithm by which an agent plans what actions curiosity) in a flexible learning agent. judgment on the question of how well rationality
to take conditional on its perceptions. Compu- Although the mantle of designing machina theories capture essential human behavior, we
tation is abstracted away in the standard eco- economicus may not be adopted (particularly note the irony in the prospect that social science
nomic model and is precisely what the AI scientist in such explicit terms) by all AI researchers, theories may turn out to apply with greater fidel-
must account for to operationalize rationality in a many AI advances over the past few decades ity to nonhuman agent behavior.
realized agent. can be characterized as progress in operation-
This does not mean that an AI design needs to alizing rationality. For instance, probabilistic Reasoning about other agents
incorporate data structures corresponding directly reasoning was largely eschewed by AI 30 years The issue of agent theorizing is not merely aca-
to rationality constructs, although many AI archi- ago but now pervades the field, thanks to demic. If we can build one AI agent, then we can
tectures do feature direct representations for prop- developments in representation and inference build many, and these AIs will need to reason
ositions, goals, and the like. Such representations using Bayesian networks and related graphi- about each other as well as about people. For AIs
may simplify the analysis of AI systems—for exam- cal formalisms. Expressing uncertainty about designed to approximate machina economicus, it
ple, we can ask whether an inference algorithm general relationships, beyond mere proposi- stands to reason that they should treat each other
operating on logical expressions possesses desir- tions, is routinely supported in probabilistic as rational, at least as a baseline assumption. These
able properties such as soundness: that all con- modeling languages (16). Statistical approaches AIs would adopt a game-theoretic view of the
clusions follow from the premises. Similarly, if an now dominate machine learning and natural world, where agents rationally respond to each
AI’s beliefs are encoded as probability distribu- language processing (17, 18). Likewise, prefer- others’ behavior, presumed (recursively) to be ra-
tions, we can ask whether it updates its beliefs ence handling (including methods for eliciting tional as well. A consequence is that agents would
from observations in proper accord with Bayesian preferences from the designer of an AI agent, expect their joint decisions to be in some form of
theory. However, care must be taken in under- compactly representing preferences over com- equilibrium, as in standard economic thinking.
Varian (36) has argued that the theory of to bid amount (as well as other factors, such as The tangle between automated agents and the
mechanism design may actually prove more rel- ad quality), with higher-ranked ads receiving a design of rules of interaction also features prom-
evant for artificial agents than for human agents, higher position on the search results page. Early inently in today’s financial markets, where the
because AIs may better respect the idealized auction mechanisms employed first-price rules, dominance of computerized traders has, by most
assumptions of rationality made in this frame- charging an advertiser its bid amount when its accounts, qualitatively shaped the behavior of
work. For example, one desirable property of a ad receives a click. Recognizing this, advertisers these markets. Although details of implementa-
mechanism is incentive compatibility, which stip- employed AIs to monitor queries of interest, tion are closely held secrets, it is well understood
ulates that truthful reports constitute an equi- ordered to bid as little as possible to hold onto that techniques from AI and machine learning
librium. Sometimes it is even possible to make the current position. This practice led to cascades are widely employed in the design and analysis
truthful reporting a dominant strategy (optimal of responses in the form of bidding wars, amounting of algorithmic traders (66). Algorithmic trading
whatever others do), achieving the strong prop- to a waste of computation and market ineffi- has enabled the deployment of strategies that
erty of strategy-proofness (37). It seems, however, ciency (55). To combat this, search engines in- exploit speed advantages and has led in turn to a
that people do not reliably understand this pro- troduced second-price auction mechanisms (37), costly arms race of measures to respond to market
perty; evidence from medical matching markets, which charge advertisers based on the next-highest information with minimum latency. A proposed
and also from laboratory experiments, suggests bid price rather than their own price. This ap- design response would replace continuous-time
that some participants in strategy-proof matching proach (a standard idea of mechanism design) auctions with periodic auctions that clear on the
mechanisms try to misrepresent their preferences removed the need to continually monitor the bid- order of once per second, thus negating the
even though it provides no advantage (38, 39). ding to get the best price for position, thereby end- advantage of tiny speed improvements (67, 68).
For artificial systems, in comparison, we might ing bidding wars (56). We describe two additional examples of the
expect AIs to be truthful where this is optimal In recent years, search engine auctions have design of multi-agent systems for an economy
and to avoid spending computation reasoning supported richer, goal-based bidding languages. of AIs. The first example system aggregates
particularly in situations where multiple equilib- 19. Y. Shoham, K. Leyton-Brown, Multiagent Systems: Algorithmic, 55. B. Edelman, M. Ostrovsky, Decis. Support Syst. 43, 192–198
ria exist. As Shoham (83) argues, game theory Game-Theoretic and Logical Foundations (Cambridge Univ. (2007).
Press, Cambridge, 2009). 56. B. Edelman, M. Ostrovsky, M. Schwarz, Am. Econ. Rev. 97,
lacks a well-accepted pragmatic account of how 20. D. Fudenberg, D. K. Levine, The Theory of Learning in Games 242–259 (2007).
it should be deployed in concrete reasoning con- (MIT Press, Cambridge, 1998). 57. C. Borgs et al., 16th International Conference on World-Wide
texts. A positive view is that AI researchers, in 21. M. P. Wellman, D. M. Reeves, K. M. Lochner, Y. Vorobeychik, Web (2007), pp. 531–540.
their efforts to operationalize economic reason- J. Artif. Intell. Res. 21, 19–36 (2004). 58. J. Feldman, S. Muthukrishnan, M. Pál, C. Stein, 8th
22. D. Billings, A. Davidson, J. Schaeffer, D. Szafron, Artif. Intell. ACM Conference on Electronic Commerce (2007),
ing, are developing exactly this needed body of 134, 201–240 (2002). pp. 40–49.
pragmatics. 23. M. Bowling, N. Burch, M. Johanson, O. Tammelin, Science 347, 59. A. Z. Broder, E. Gabrilovich, V. Josifovski, G. Mavromatis,
Some may object that mechanism design is too 145–149 (2015). A. J. Smola, 4th International Conference on Web Search
idealized even for systems of AIs—for example, in 24. T. Sandholm, AI Mag. 31, 13–32 (2010). and Web Data Mining (2011), pp. 515–524.
25. N. Brown, S. Ganzfried, T. Sandholm, 14th International 60. L. Hurwicz, in Decision and Organization, R. Radner,
its insistence on design under equilibrium behav- Conference on Autonomous Agents and Multi-Agent Systems C. B. McGuire, Eds. (North-Holland, Amsterdam, 1972),
ior, its assumption that rules of interaction can be (2015), pp. 7–15. Ch. 14, pp. 297–336.
designed from scratch, and its lack of attention to 26. R. Gibson, Regret minimization in games and the development 61. A. Gibbard, Econometrica 41, 587–602 (1973).
the details of the human and legal contexts in of champion multiplayer computer poker-playing agents, 62. R. Myerson, Econometrica 47, 61–73 (1979).
Ph.D. thesis, University of Alberta (2014). 63. E. Clarke, Public Choice 11, 17–33 (1971).
which designed systems will operate. A positive 27. M. Tambe, Security and Game Theory: Algorithms, Deployed 64. T. Groves, Econometrica 41, 617–631 (1973).
view is that AI systems are precisely the kinds of Systems, Lessons Learned (Cambridge Univ. Press, Cambridge, 65. H. R. Varian, C. Harris, Am. Econ. Rev. 104, 442–445
environments where we can build tabula rasa 2011). (2014).
new rules of interaction, because these rules will 28. Y. Gal, A. Pfeffer, J. Artif. Intell. Res. 33, 109–147 (2008). 66. M. Kearns, Y. Nevmyvaka, in High Frequency Trading:
29. C. F. Camerer, Behavioral Game Theory: Experiments in New Realities for Traders, Markets and Regulators, D. Easley,
be realized through the Internet and as programs Strategic Interaction (Princeton Univ. Press, Princeton, M. Lopez de Prado, M. O’Hara, Eds. (Risk Books, London,
running on computer servers. That such rules of 2003). 2013), Ch. 5, pp. 91–124.
interaction can come into existence is as much a 30. R. Yang, C. Kiekintveld, F. Ordóñez, M. Tambe, R. John, Artif. 67. E. Budish, P. Cramton, J. Shim, The high-frequency trading
RELATED http://science.sciencemag.org/content/sci/349/6245/248.full
CONTENT
REFERENCES This article cites 42 articles, 3 of which you can access for free
http://science.sciencemag.org/content/349/6245/267#BIBL
PERMISSIONS http://www.sciencemag.org/help/reprints-and-permissions
Science (print ISSN 0036-8075; online ISSN 1095-9203) is published by the American Association for the Advancement of
Science, 1200 New York Avenue NW, Washington, DC 20005. 2017 © The Authors, some rights reserved; exclusive
licensee American Association for the Advancement of Science. No claim to original U.S. Government Works. The title
Science is a registered trademark of AAAS.