A Bayesian Model of Conditioned PerceptionAlan A. Stocker∗and Eero P. SimoncelliHoward Hughes Medical Institute,Center for Neural Science,and Courant Institute of Mathematical SciencesNew York UniversityNew York, NY-10003, U.S.A.presented at: NIPS 21, Vancouver BC Canada, December 5th, 2007.to appear in: Advances in Neural Information Processing Systemsvol 20, pp XXX-XXX, May 2008MIT Press, Cambridge MA.We argue that in many circumstances, human observers evaluate sensory evidencesimultaneously under multiple hypotheses regarding the physical process that hasgenerated the sensory information. In such situations, inference can be optimal ifan observer combines the evaluation results under each hypothesis according tothe probability that the associated hypothesis is correct. However, a number of ex-perimental results reveal suboptimal behavior and may be explained by assumingthat once an observer has committed to a particular hypothesis, subsequent evalu-ation is based on that hypothesis alone. That is, observers sacrifice optimality inorder to ensure self-consistency. We formulate this behavior using a conditionalBayesian observer model, and demonstrate that it can account for psychophysicaldata from a recently reported perceptual experiment in which strong biases in per-ceptual estimates arise as a consequence of a preceding decision. Not only doesthe model provide quantitative predictions of subjective responses in variants ofthe original experiment, but it also appears to be consistent with human responsesto cognitive dissonance.1 MotivationIs the glass half full or half empty? Indifferentsituations, the verysameperceptualevidence(e.g. theperceived level of liquid in a glass) can be interpreted very differently. Our perception is conditionedon the context within which we judge the evidence. Perhaps we witnessed the process of the glassbeing filled, and thus would more naturally think of it as half full. Maybe it is the only glass onthe table that has liquid remaining, and thus its precious content would be regarded as half full. Ormaybe we simply like the content so much that we cannot have enough, in which case we may viewit as being half empty.Contextual influences in low-level human perception are the norm rather than the exception, andhave been widely reported. Perceptual illusions, for example, often exhibit particularly strong con-textual effects, either in terms of perceptual space (e.g. spatial context affects perceived brightness;see [1] for impressive examples) or time (prolonged exposure to an adaptor stimulus will affectsubsequent perception, see e.g. the motion after-effect [2]). Data of recent psychophysical exper-iments suggest that an observer’s previous perceptual decisions provide additional form of contextthat can substantially influence subsequent perception [3, 4]. In particular, the outcome of a categor-ical decision task can strongly bias a subsequent estimation task that is based on the same stimuluspresentation. Contextual influences are typically strongest when the sensory evidence is most am-biguous in terms of its interpretation, as in the example of the half-full (or half-empty) glass.Bayesian estimators have proven successful in modeling human behavior in a wide variety of low-level perceptual tasks (for example: cue-integration (see e.g. [5]), color perception (e.g. [6]), visualmotion estimation (e.g. [7, 8])). But they generally do not incorporate contextual dependenciesbeyond a prior distribution (reflecting past experience) over the variable of interest. Contextualdependencies may be incorporated in a Bayesian framework by assuming that human observers,when performing a perceptual task, test different hypotheses about the underlying structure of the∗corresponding author.sensory evidence, and arrive at an estimate by weighting the estimates under each hypothesis ac-cording to the strength of their belief in that hypothesis. This approach is known as optimal modelevaluation [9], or Bayesian model averaging [10] and has been previously suggested to account forcognitive reasoning [11]. It further has been suggested that the brain could use different neuro-modulators to keep track of the probabilities of individual hypotheses [12]. Contextual effects arereflected in the observer’s selection and evaluation of these hypotheses, and thus vary with exper-imental conditions. For the particular case of cue-integration, Bayesian model averaging has beenproposed and tested against data [13, 14], suggesting that some of the observed non-linearities incue integration are the result of the human perceptual system taking into account multiple potentialcontextual dependencies.In contrast to these studies, however, we propose that model averaging behavior is abandoned oncethe observer has committed to a particular hypothesis. Specifically, subsequent perception is condi-tioned only on the chosen hypothesis, thus sacrificing optimality in order to achieve self-consistency.We examine this hypothesis in the context of a recent experiment in which subjects were asked toestimate the direction of motion of random dot patterns after being forced to make a categoricaldecision about whether the direction of motion fell on one side or the other of a reference mark [4].Depending on the different levels of motion coherence, responses on the estimation task were heav-ily biased by the categorical decision. We demonstrate that a self-consistent conditional Bayesianmodel can account for mean behavior, as well as behavior on individual trials [8]. The model has es-sentially no free parameters, and in addition is able to make precise predictions under a wide varietyof alternative experimental arrangements. We provide two such example predictions.2 Observer ModelWe define perception as a statistical estimation problem in which an observer tries to infer the valueof some environmental variable s based on sensory evidence m (see Fig. 1). Typically, there aresources of uncertainty associated with m, including both sensor noise and uncertainty about therelationship between the sensory evidence and the variable s. We refer to the latter as structuraluncertainty which represents the degree of ambiguity in the observer’s interpretation of the physicalworld. In cases where the structural possibilities are discrete, we denote them as a set of hypothesesH = {h1, ..., hN}. Perceptual inference requires two steps. First, the observer computes their beliefprior