Title:  Probabilistic models of value for decisions during action

Paul Schrater

University of Minnesota

Abstract: While it is fair to say we choose what we value, the relative ease with which we make choices and actions masks deep uncertainties and paradoxes in our representation of value. For example, ambiguous and uncertain options are typically devalued when pitted against sure things - however, curiosity makes uncertainty valuable. In general, ecological decisions can involve goal uncertainty, uncertainty about the value of goals, and time/state-dependent values. When a soccer player moves the ball down the field, looking for an open teammate or a chance to score a goal, the value of action plans like passing, continuing or shooting depends on conditions like teammate quality, remaining metabolic energy, defender status and proximity to goal all of which need to be integrated in

real time.  We show how probabilistic representations of value can solve the problem of converting and integrating heterogeneous values, like metabolic costs vs. scoring a soccer goal. By modeling values in terms of probabilities of achieving better outcomes, we decompose complex problems like the soccer player into weighted mixture of control policies, each of which produces a sequence of actions associated with more specific goal. Critically, the weights are inferences that integration all the time-varying probabilistic information about the relative quality of each policy. We use the approach to give a rational account for a set of reaching and oculomotor experiments

with multiple goals.