When you see a glass fall off the table, you can predict it will break without seeing it hit the ground. Similarly, if you hear the glass shatter, you can infer what happened without seeing anything at all. Our knowledge of the causal mechanisms structuring our world allows us to make impressively accurate inferences based on incomplete information spread across multiple sensory modalities. In this work, we study the cognitive processing that supports this remarkable behavior. We utilize the Plinko domain, an intuitive physics setup where marbles are dropped into a box from one of three holes, colliding with obstacles as they fall to the ground. Participants judge where they think the ball fell from based on visual and auditory evidence. We track participants' eye-gaze to gain deeper insight into their mental processes, and develop models that characterize the computational processes underlying participant behavior.

People use varied language to express their causal understand-ing of the world. But how does that language map onto peo-ple’s underlying representations, and how do people choosebetween competing ways to best describe what happened? Inthis paper we develop a model that integrates computationaltools for causal judgment and pragmatic inference to addressthese questions. The model has three components: a causalinference component which computes counterfactual simula-tions that capture whether and how a candidate cause madea difference to the outcome, a literal semantics that mapsthe outcome of these counterfactual simulations onto differentcausal expressions (such as “caused”, “enabled”, “affected”,or “made no difference”), and a pragmatics component thatconsiders how informative each causal expression would befor figuring out what happened. We test our model in an ex-periment that asks participants to select which expression bestdescribes what happened in video clips depicting physical in-teractions.

Mental simulation is a powerful cognitive capacity that underlies people's ability to draw inferences about what happened in the past from the present. Recent work suggests that eye-tracking can be used as a window through which one can study the process of mental simulation in intuitive physics tasks. In our experiment, participants have to figure out in which of three holes a ball was dropped in a virtual Plinko box. We develop a computational model of human intuitive physical reasoning in Plinko that runs repeated simulations in a noisy physics simulator in order to infer in which hole the ball was dropped. We evaluate our model's behavior against multiple human data signals: trial judgments, response times, and eye-movement data. We find that a model that sequentially samples simulations while balancing uncertainty and reward best explains the patterns of participant behavior we observe in these three signals.